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Preface 



This volume contains the proceedings of the first joint PAPM-PROBMIV Work- 
shop, held at the Rheinisch-Westfalische Technische Hochschule (RWTH) Aachen, 
Germany, 12-14 September 2001. 

The PAPM-PROBMIV workshop results from the combination of two work- 
shops: PAPM (Process Algebras and Performance Modeling) and PROBMIV 
(Probabilistic Methods in Verification). The aim of the joint workshop is to 
bring together the researchers working across the whole spectrum of techniques 
for the modeling, specification, analysis, and verification of probabilistic systems. 
Probability is widely used in the design and analysis of software and hardware 
systems, as a means to derive efficient algorithms (e.g. randomization), as a 
model for unreliable or unpredictable behavior (as in the study of fault-tolerant 
systems and computer networks), and as a tool to study performance and de- 
pendability properties. The topics of the workshop include specification, mod- 
els and semantics of probabilistic systems, analysis and verification techniques, 
probabilistic methods for the verification of non-probabilistic systems, and tools 
and case studies. 

The first PAPM workshop was held in Edinburgh in 1993; the following ones 
were held in Regensberg (1994), Edinburgh (1995), Torino (1996), Enschede 
(1997), Nice (1998), Zaragoza (1999), and Geneva (2000). The first PROBMIV 
workshop was held in Indianapolis, Indiana (1998); the next one took place in 
Eindhoven (1999). In 2000, PROBMIV was replaced by a Dagstuhl seminar on 
Probabilistic Methods in Verification. 

The PAPM-PROBMIV workshop is held in conjunction with two other work- 
shops: 11th GI/ITG Gonference on Measuring, Modeling, and Evaluation of 
Gomputer and Gommunications Systems (MMB), and the 9th International 
Workshop on Petri Nets and Performance Models (PNPM). Together, these three 
workshops form the 2001 Aachen Multiconference on Measurement, Modeling, 
and Evaluation of Computer-Communication Systems. We hope that this setting 
fosters the exchange of ideas with neighboring research fields and allows for a 
comparison of different viewpoints towards similar problems. 

Of the 23 regular papers, 12 were accepted for presentation at the workshop 
and are included in the present volume. The workshop is preceded by three 
tutorials, given by Joost-Pieter Katoen (University of Twente) on Probabilistic 
verification of Markov chains, by Marina Ribaudo (University of Torino) on An 
introduction to stochastic process algebras, and by Roberto Segala (University 
of Bologna) on Nondeterminism in probabilistic verification. The workshop in- 
cludes three invited presentations, by Shankar Sastry (University of Galifornia, 
Berkeley), Markus Siegle (Friedrich- Alexander Universitat Erlangen-Niirnberg), 
and Frits Vaandrager (University of Nijmegen). 

We thank all the members of the program committee, and their sub-referees, 
for selecting the papers to be presented. Special thanks are due to Boudewijn 
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Haverkort (University of Aachen), the general chair of the multi-conference and 
local organization, and Peter Kemper (University of Dortmund), the tool ses- 
sion chair. Our thanks go to the following organizations for their generous spon- 
sorship of the Aachen multiconference: German Research Association (DFG), 
IBM Deutschland, Siemens AG Miinchen (Information and Gommunication Net- 
works), T-Nova Deutsche Telekom Innovationsgesellschaft mbH, and TENOVIS. 
Our thanks also go to all the authors for meeting the tight deadlines which we 
set without compromising on the rigor or clarity of their papers. 
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Abstract. We review high-level specification formalisms for Markovian 
performability models, thereby emphasising the role of structuring con- 
cepts as realised par excellence by stochastic process algebras. Symbolic 
representations based on decision diagrams are presented, and it is shown 
that they quite ideally support compositional model construction and 
analysis. 



1 Introduction 

Stochastic models have a long tradition in the areas of performance and depend- 
ability evaluation. Since their specification at the level of the Markov chain is 
tedious and error-prone, several high-level model specification formalisms have 
been developed, such as queueing networks, stochastic Petri nets and networks 
of stochastic automata, which allow humans to describe the intended behaviour 
at a convenient level of abstraction. Although under Markovian assumptions 
the analysis of the underlying stochastic process does not pose any conceptual 
problems, the size of the underlying state space often renders models intractable 
in practice. Structuring concepts have shown to be of great value in order to 
alleviate this well-known state space explosion problem. 

A process algebra is a mathematically founded specification formalism which 
provides compositional features, such as parallel composition of components, 
abstraction from internal actions, and the replacing of components by be- 
haviourally equivalent ones. Therefore, stochastic extensions of process algebras 
are among the methods of choice for constructing complex, hierarchically struc- 
tured stochastic models. 

Recently, decision diagrams, which were originally developed as memory- 
efficient representations of Boolean functions in the area of hardware verifica- 
tion, have been extended in order to capture the numerical information which 
is contained in stochastic models. They have already been successfully used as 
the underlying data structure in prototype tools for performance analysis and 
verification of probabilistic systems. In this paper, it is shown that symbolic rep- 
resentations based on decision diagrams are particularly attractive if applied in 
a compositional context, as provided, for example, by a process algebraic spec- 
ification formalism. In many cases, decision diagrams allow extremely compact 
representations of huge state spaces, and it has been demonstrated that all steps 
of model construction, manipulation and analysis (be it model checking, numer- 
ical analysis, or a combination of the two) can be carried out on the decision 
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diagram based representations. Thus, we argue that decision diagrams fit in well 
with structured modelling formalisms and open new ways towards increasing the 
range of manageable performance evaluation and verification problems. 

This paper does not intend to present new research results, but to survey the 
history of structured model representations, with special emphasis on process 
algebras and symbolic encodings. We provide many pointers to further reading, 
without attempting to be exhaustive. 

The paper is organised as follows: In Sec. |2 we survey the evolution from 
monolithic to modular model specification formalisms. Sec.0reviews the concept 
of stochastic process algebras. Sec. 21 introduces the symbolic representation of 
Markovian models with the help of multi-terminal binary decision diagrams and 
describes compositional model construction, manipulation and analysis on the 
basis of this data structure. The paper concludes with Sec. 0 

2 Prom Unstructured to Structured Models 

2.1 Monolithic Model Representations 

Continuous Time Markov Chains (CTMC) are the basic formalism for specifying 
performance and dependability model^. A CTMC consists of a (finite, for our 
purpose) set of states and a finite set of transitions between states. The transi- 
tions are labelled by positive reals, called the transition rates, which determine 
transition probabilities and state sojourn times (the latter being exponentially 
distributed). Time-dependent state probabilities can be derived by solving a 
system of ordinary differential equations, and steady-state probabilities are cal- 
culated by solving a linear system of equations (see, for instance |Ji|). In order 
to save memory space, CTMCs are commonly represented as sparse matrices, 
where essentially only the non-zero entries are stored. 

The direct specification of a CTMC at the level of individual states and state- 
to-state transitions is tedious and error-prone, and therefore only feasible for 
very small models. This motivated researchers to develop high-level specification 
formalisms for defining Markovian models at a level of abstraction which is more 
convenient for the human modeller. The most popular of these formalisms are 
queueing networks and stochastic Petri nets. 

Queueing networks (QN), developed mainly in the 1960ies and 1970ies for 
modelling time-sharing and polling systems, describe customers moving between 
stations where they receive service after possibly waiting for a service unit to 
become available. The aim of analysis is typically the mean or distribution of 
the number of customers at a station, the customer throughput at a station, or 
the waiting time. The success of queueing networks stems mainly from the fact 
that for the class of product form networks jS| very efficient analysis algorithms, 
such as Buzen’s algorithm m or mean- value analysis m, are known, and that 
software tools for the specification and analysis of QN models were available at 

^ In this paper, we do not consider the line of research on non-Markovian models such 
as described, for example, in m 
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an early stage mm- Although QN have been extended in various directions, 
e.g. in order to model the forking and synchronisation of jobs (fork-join QNs, 
p3lti8l80| l. the formalism of QNs is not suitable for the modelling of arbitrary 
systems, but specialised to the application area of shared resource systems. 



Stochastic Petri nets (SPN) were developed in the 1980ies for modelling 
complex synchronisation schemes which cannot easily be expressed by queueing 
models izg. The modelling primitives of Petri nets (places, transitions, mark- 
ings) are very basic and do not carry any application-specific semantics. For 
that reason, Petri nets are universally applicable and very flexible, which is re- 
flected by the fact that they have been successfully applied to many different 
areas of application. In the class of generalised SPNs (GSPN) [DEI, transitions 
are either timed or immediate. Timed transitions are associated with an expo- 
nentially distributed firing time, while immediate transitions Are as soon as they 
are enabled. During the analysis of a GSPN, the reachability graph is generated 
and the so-called vanishing markings, which are due to the firing of immediate 
transitions, are eliminated. The result is a GTMG whose analysis yields (steady- 
state or transient) state probabilities, i.e. the probabilities of the individual net 
markings, from which high-level measures can be computed. 

Some software tools for performance modelling, e.g. USENUM |^, MARGA 
MOSEL [3| and DNAmaca EDI, implement their own specialised model 
description languages, which can also be considered as high-level specification 
formalisms for GTMGs. 



With the help of the high-level model specification formalisms considered 
so far it is possible to specify larger GTMGs than at the state-to-state level, 
but these formalisms do not support the concepts of modularity, hierarchy or 
composition of submodels. As a result, the models are monolithic and may be 
difficult to understand and debug. Moreover, state space generation and numer- 
ical analysis of very large monolithic GTMGs is often not feasible in practice 
due to memory and GPU time limitations, which is referred to as the notorious 
state space explosion problem. 

A large state space may become tractable if it is decomposed into smaller 
parts Instead of analysing one large system, the decomposition approach 

relies on analysing several small subsystems, analysing an aggregated overall sys- 
tem, and afterwards combining the subsystems’ solutions accordingly. In general, 
this approach works well for nearly completely decomposable (NGD) systems 
whose state space can be partitioned into disjoint subsets of states, such that 
there is a lot of interaction between states belonging to the same subset, but 
little interaction between states belonging to different subsets. For the class of 
reversible Markov chains, the decomposition/aggregation approach yields ex- 
act results m We mention that the approach may also be applied iteratively 
The major question is, of course, how to best partition a given state 
space, and in general this information should be derived from a modular high- 
level model specification. Approximate decomposition-based analysis methods 
for stochastic process algebra models (see Sec. 0 are discussed in ca, where 
time scale decomposition is based on the concept of NGD Markov chains [POirz!j . 
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and response time approximation relies on a structural decomposition for the 
special class of decision-free processes m- Another such approach, based on the 
exploitation of the structure of a special class of process algebraic models, is de- 
scribed in |7|. Approximate decomposition-based analysis for nearly-independent 
GSPN structures is considered in EZEOI- 



2.2 Modular Model Representations 



Queueing models, stochastic Petri nets and the tool-specific modelling languages 
mentioned above do not offer the possibility of composing an overall model from 
components which can be specified in isolation. Such a composition, however, 
is a highly desirable feature when modelling complex systems, since it enables 
human users to focus on manageable parts from which a whole system can be 
constructed. For instance, modern performance analysis advocates a separation 
of the load model and the machine model, an idea developed already in EHEa 
Itilil . and similar ideas are also applied in stochastic rendezvous networks 
and layered queueing networks |H2|. As another, specific example, suppose one 
wished to model a communication system where two partners communicate over 
some communication medium. The model should reflect this structure, i.e. it 
should consist of three interacting submodels, one for each partner, and one for 
the medium. The user should be able to specify these three submodels more or 
less independently of each other and then simply specify the way in which they 
interact. 

In the basic GSPN formalism, a model consists of a single net which covers 
the whole system to be studied. Therefore, GSPN models of complex systems 
tend to become very large and confused and suffer from the state space explo- 
sion problem. Stochastic activity networks jiSSI-i5] constitute an approach to the 



structuring of GSPNs through the sharing of places between different subnets. In 
the presence of symmetric submodels, they tackle the state space explosion prob- 
lem by directly generating a reduced reachability graph in which all mutually 
symmetric markings are collapsed into one. Symmetries also play a predominant 
role for the analysis of stochastic well- formed coloured Petri nets j2til4,'-ij . where 
a reduced reachability graph is constructed directly from the net description, 
without the need to construct the full reachability graph first. Another line of 
research is concerned with building SPNs in a structured way, basically by syn- 
chronising subnets via common transitions, which is an instance of the Kronecker 
approach described below. 

Stochastic automata networks (SAN)0, developed in the 1980ies and 1990ies 
[liS2i;s;-il?S4l?St)) . consist of several stochastic automata, basically GTMGs whose 
transitions are labelled with event names, which run in parallel and may per- 
form certain synchronising events together. Thus, the SAN formalism is truly 
structured, since it allows the user to specify an overall model as a collection of 
interacting submodels. The major attraction of SANs is their memory-efficient 



^ The acronym SAN is also used for stochastic activity networks (see above), but in 
this paper it stands for stochastic automata networks. 
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representation of the generator matrix of the Markov chain underlying the over- 
all model. This so-called Kronecker (or tensor) approach has since been adapted 
to queueing networks ca. stochastic Petri nets stochastic 

process algebras CHI and other structured modelling frameworks IPi'i'itlllll'il . 

The Kronecker approach realises an implicit, space-efficient representation of 
the transition rate matrix of a stuctured Markov model. Suppose we have two 
independent CTMCs C\ and C2 which are given by their transition rate matrices 
R\ and i?2 (of size di and d^). Let us consider the combined stochastic process 
C whose state space is the Cartesian product of the state spaces of C\ and C^- 
Process C possesses the transition rate matrix R which is given by the Kronecker 
sum of R\ and i?2: 



R — R\ © R 2 — Rl © Id.2 © ^di © 4?2 

where © denotes Kronecker product, ® denotes Kronecker sum and Id denotes 
an identity matrix of size d m- If, however, C\ and C2 are not independent, 
but perform certain transitions synchronously, the expression for the overall 
transition rate matrix changes to 

R — Rl,i ® 4?2,i + 'y ^ Aa • R\^a © 4?2,o 

aes 

where Ri^ and i?2,i contain those transitions which C\ and C2 perform inde- 
pendently of each other, and R\^a and i?2,a contain those transitions which are 
caused by an event a from the set of synchronising events S. Here it is assumed 
that the resulting rate of the synchronising event a is given by Aa, i.e. it is 
a predetermined rate, and matrices Ri^a and i?2,a are indicator matrices which 
contain only zeroes and ones. (It is also possible that i?i,a and i?2,a contain rates, 
in which case in the above subexpression A^ • R\^a © R2,a has to be replaced by 
Ri,a © 4 ? 2 .a- This would mean that the resulting rate of a synchronising event is 
equal to the product of the rates of the participating processes.) For the general 
case, where the overall model consists of K submodels, the expression for the 
overall transition rate matrix is given by 

K K 

R = Rk,i + ^ Aa ■ Rk,a 

k—1 a^S k—1 

The strength of the Kronecker approach lies in its memory-efficiency (it suffices 
to store a set of matrices of the size of the submodels) and in the fact that for 
performing numerical analysis, the potentially very large overall transition rate 
matrix never needs to be constructed or stored explicitly. The compactness of 
the representation of the transition rate matrix carries over to the generator 
matrix and to the iteration matrices for some of the common stationary iter- 
ative methods. Thus, iterative numerical schemes which rely on matrix-vector 
multiplication as their basic operation, can be performed directly on the ten- 
sor descriptor of the iteration matrix (Plateau used the power method, and 
Buchholz uni describes Kronecker-based power, Jacobi, modified Gauss-Seidel, 
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JOR and modified SOR methods). Efficient algorithms for the multiplication of 
a vector with a Kronecker descriptor are analysed in and in EDI, where, 

however, the authors state that . all Kronecker-based algorithms are less com- 
putationally efficient than a conventional multiplication where [the matrix] R ist 
stored in sparse format ...” and “This suggests that, in practice, the real advan- 
tage of Kronecker-based methods lies exclusively in their large memory savings” . 

When working with the Kronecker approach, the set of states reachable from 
the initial state may be only a small subset of the Cartesian product of the 
involved submodel state spaces. This is known as the “potential versus actual 
state space” problem. If the actual state space is not known before numerical 
analysis starts, a probability vector of the size of the potential state space must 
be allocated, which can waste a considerable amount of memory space and even 
make the whole analysis impracticable. For that reason, Kronecker-based reach- 
ability techniques have been developed, which allow one to work on the actual 
state space or a limited superset thereof 



[an ttzi 



3 Stochastic Process Algebras 



In this section, we briefly review the concept of stochastic process algebras (SPA). 
Since process algebras feature composition operators that allow one to construct 
complex specifications from smaller ones, we argue that they quite ideally sup- 
port the specification and analysis of structured models. Next we define a simple 
SPA language which supports both Markovian and immediate transitions. 

Definition 1. Stochastic process algebra language £ 

Let Act be the set of valid aetion names and Pro the set of proeess names. Let 
aetion r G Act denote the internal, invisible aetion. For P, Pi £ C, a G Act, 
S C Act \ {r}, and X £ Pro, the set C of valid expressions is definded by the 
following language elements: 



stop inaction 

a; P immediate prefix 

Pi + P2 choice 

hide a in P hiding 



(a. A); P Markovian prefix 

Pi I [S'] I P2 parallel composition 

X process instantiation 



A set of definitions of the form X := P eonstitutes a proeess environment. ■ 



With the help of a structured operational semantics, a transition system whose 
states correspond to process terms can be derived as the semantic model of 
a process algebraic specification. For a discussion of the full set of semantic 
rules for Markovian process algebras similar to our language C we refer the 
interested reader to the literature, see e.g. |6I48I53I55I64| . Here we only mention 
two selected rules. The first is the rule for synchronisation of two processes via 
Markovian transitions which can be written as follows: 



P 



b,A 



P' Q^Q' 



b£S 






b, 4 >{\ 11) 



P'\[S]\Q' 
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Note that this rule is parametric in a function (j) determining the rate of synchro- 
nisation, since different synchronisation policies (minimum, maximum, product, 

. . . ) are possible. In the process algebra TIPP I5D], 4> is instantiated by multi- 
plication, since strong bisimilarity (see below) is a congruence with respect to 
parallel composition and abstraction, provided that (j) is distributive over sum- 
mation of real values, see . Note that the apparent rate construction of 

PEPA 1 ^ requires a function cj){P, Q, A, fi) instead of </>(A, ^). 

The second rule is the one for hiding in the case of immediate transitions, 
which states that an immediate transition labelled by a is turned into an internal 
immediate transition labelled by r: 

P P' 

hide a in P hide a in P' 

As we shall see, internal immediate transitions, as generated by this rule, play 
a key role during the transformation from a transition system to a CTMC. For 
our stochastic process algebra language C, the resulting semantic model is an 
extended stochastic labelled transition system (ESLTS): 

Definition 2. Extended Stochastic Labelled Transition System (ESLTS) 

Let S be a finite set of states. Let sq € S be the initial state. Let Act be a finite 
set of aetion labels. Let — ^ be defined as follows: 

— ^ C S X Act X S 

Let — > be defined as follows: 

— ^ C S X Act X 1R>° X S 

We eall T = {S,Act,—^, — t,so) o,n Extended Stochastic Labelled Transition 
System. Lf(x,a,y) € — we say that there is an immediate a-transition from 
state X to state y and write x — + y. Lf {x, b, A, y) G — >, we say that there is a 
Markovian b-transition from state x to state y with rate A and write x y. ■ 

Note that in view of the symbolic representation described below, we restricted 
ourselves to finite-state transition systems. An ESLTS whose set of immediate 
transitions is empty is called SLTS. An example ESLTS is depicted in Fig. 0 
(left). Basically, immediate transitions lead to the existence of vanishing (insta- 
ble) states. These are states which are left as soon as they are entered, i.e. their 
sojourn time is zero. Conversely, tangible (stable) states are states whose sojourn 
time has an exponential distribution, i.e. is strictly positive. For the performa- 
bility analysis of an SPA model, a CTMC is constructed from the ESLTS and 
analysed with conventional numerical methods. The CTMC is obtained by hid- 
ing of all action labels, elimination of the vanishing states and proper cumulation 
of all Markovian transitions between a given ordered pair of states. 

For a compositional framework, as in the context of stochastic process al- 
gebras, we propose to refine the well-known notion of vanishing states in the 
following way: 
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Fig. 1. Role of visible immediate transitions during parallel composition 
Definition 3. Compositionally vanishing states 

A state s of an ESLTS is called vanishing if there is at least one internal im- 
mediate transition emanating from s (written s — ^ s'). A state s of an ESLTS 
is called compositionally vanishing if it is vanishing and if there is no visible 
immediate transition emanating from s (written s s" , where t). ■ 

The idea is that even an immediate transition may be delayed if it is visible, 
since it may be kept waiting by a synchronisation partner which is not yet 
ready to participate in the synchronisation. Since synchronisation on internal 
r-transitions is not allowed, one can be sure that internal immediate transitions 
will not be delayed. Compositionally vanishing states can be eliminated either 
before or after composition of subprocesses, but a vanishing state that may also 
be left by at least one visible immediate transition must not be eliminated before 
compositiorJ§, An example for such a situation is shown in Fig. Q which shows 
two ESLTSs, 7 i and ?2, which are composed in parallel, synchronising on action 
c. The resulting ESLTS, T, is shown on the right hand side of the figure. State si 
in ESLTS 7 i , which is vanishing but not compositionally vanishing, must not be 
eliminated before parallel composition takes place, since its elimination would 
disable any c-transition in the combined transition system T. In the resulting 
ESLTS, state sio is a compositionally vanishing state which can be eliminated, 
whereas state sn is not. However, if action c is hidden in ESLTS T (since 
further synchronisation on c is not required), state sn becomes compositionally 
vanishing and can be also eliminated (its elimination, however, requires a proper 
treatment of non-determinism as explained below) . Note also that there may be 
one or several Markovian transitions emanating from a vanishing state, but they 
are never taken. As an example, in Fig. Ethe transition sio sn will never be 

T 

taken, since the competing internal immediate transition sio — ^ S30 will always 

take place first. Therefore transition sio sn can safely be deleted without 
changing the behaviour of the ESLTS. 

® To complete the picture: A state s is called tangible if there is no immediate tran- 
sition (i.e. neither visible nor internal) emanating from s. In the remaining case 
(where there is at least one visible immediate transition, but no internal immediate 
transition emanating from s) the state is called inconclusive. 
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The basic strategy of elimination of compositionally vanishing states is to 
redirect transitions leading to such a state to its successor states. In the case 
where a compositionally vanishing state has more than one outgoing internal 
immediate transitions, it is not specified which of them will be taken. This is 
an instance of non-determinism. In order to resolve such non-determinism, one 
may assign probabilities or weights to internal immediate transitions. Transi- 
tions leading to the compositionally vanishing state can then be redirected to 
its successor states, taking into account these probabilities. 

The concept of bisimilarity is of great importance for SPAs, since it estab- 
lishes the equivalence between processes, and since it is the basis for state space 
reduction. Unfortunately, it is beyond the scope of the present paper to discuss 
bisimulation relations in detail, so we refer to the literature, e.g. 



li)N liMlHiKSei 



4 Symbolic Representations 

In this section we present space-efficient symbolic representations of transition 
systems with the help of binary decision diagrams (BDD). We review multi- 
terminal BDDs (MTBDD), also called algebraic decision diagrams, since they 
are capable of representing real-valued functions mm- 

4.1 Multi-terminal BDDs 

Let IB = {0, 1} denote the set of Boolean^ An MTBDD is a graph-based 
representation of a function / : 5?" i— >■ M. 

Definition 4. Multi-Terminal Binary Decision Diagram (MTBDD) 

Let Vars = {vi, . . . , v„} be a set of Boolean variables with a fixed total order- 
ing -< C Vars X Vars. An (ordered) Multi-Terminal Binary Decision Diagram 
over {Vars,-<) is a rooted directed aeyclie graph M = (Uert, var, else, then, value) 
defined by 

• a finite nonempty set of vertices Vert = T U NT, where T (NT ) is the set of 
terminal (non-terminal) vertices, 

• a function var : NT i— >■ Vars, 

• two edge- defining functions else : NT i— > Vert and then : NT i— > Vert, 

• a function value : T i-^ M, 
with the following constraints: 

\/x S NT : else(x) S T V var(else(a;)) var(x) 

\/x S NT : then(a;) S T V var(then(cc)) var(x) ■ 

Note that, according to Def. 0 a binary decision tree is an MTBDD. However, 
we are mainly interested in reduced MTBDDs, defined as follows: 

Definition 5. Reducedness of an MTBDD 

An MTBDD M is called reduced if and only if the following conditions hold: 

We use the real numbers 0 and 1 to represent Boolean values, since in the context 
of MTBDDs Boolean variables will be involved in arithmetic calculations. 
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1. 'ix G NT : else(a;) ^ then(a;) 

2. Vx,y € NT: a; ^ y ^ (var(x) ^ var(y)Velse(a;) ^ else(y)Vthen(x) ^ then(y)) 

3. 'ix, y GT: x ^ y ^ value(a;) ^ value(y) ■ 

The first condition states that there are no “don’t care” vertices, i.e. vertices 
with identical then- and else-successors. The second condition states that there 
are no two isomorphic non-terminal vertices, and the third condition states that 
there are no two isomorphic terminal vertices. Bryant m proposed a recursive 
procedure to reduce BDD^ that can be applied to MTBDDs as well, and from 
now on, unless otherwise stated, we assume that MTBDDs are reduced. Fig. 0 
(right) shows a reduced MTBDD. In the graphical representation, the edge from 
a vertex x to then(x) is drawn solid, and the edge from x to else(a;) is drawn 
dashed. All vertices that are drawn on one level are labelled with the same 
Boolean variable, as indicated at the left of the decision diagram. In order to 
keep the figure clear, all edges leading to the zero- valued terminal vertex are not 
drawn, i.e. every non-terminal vertex with only one outgoing edge drawn has its 
other outgoing edge leading to the zero-valued terminal vertex. 

Each MTBDD vertex unambiguously defines a real-valued function, based 
on the so-called Shannon expansion which states that 

/(Vi,. . .,V„) = (1 - Vi) • /(0,V2,.. .,V„) -t- Vi • /(1,V2, . .. ,v„) 

The terms /(O, V 2 , . . . , v„) and /(I, V 2 , . . . , v„) are called the cofactors of the 
function / with respect to the Boolean variable Vi. 

Definition 6. Function represented by an MTBDD vertex 
The real-valued funetion represented by an MTBDD vertex x G Vert is re- 
cursively defined as follows: 

• if X G T then fx = value(x), 

• else (if X G NT) /„, = (1 - var(a;)) • /eise(x) + var(x) • /then(o;) ■ 

Most times one is interested in the case where x corresponds to the MTBDD 
root. In that case we write /m instead of fx, where x is the root vertex of 
MTBDD M . The two subgraphs of MTBDD M corresponding to the cofactors of 
/m are denoted and M®'®®, where M*^®” represents /m(1, V 2 , . . . , v„) and M®'®® 
represents /m(0, V 2 , . . . , v„). For a fixed ordering of Boolean variables, reduced 
MTBDDs form a canonical representation of real- valued functions, i.e. if M, M' 
are two reduced MTBDDs over the same ordered set of Boolean variables V ars 
such that /m = /mg then M and M' are isomorphic. 

It should be noted that, given a Boolean function, the size of the resulting 
MTBDD is highly dependent on the chosen variable ordering. As a prominent 
example, consider the function /id = 0^=1 which can be interpreted as 

an identity matrix of size 2". Under the interleaved variable ordering Si ^ ti ^ 

. . . ^ s„ ^ t„ the number of vertices needed to represent this function is 3n -|- 2, 
i.e. logarithmic in the size of the matrix. In contrast, using the straight-forward 

A BDD is an MTBDD where ix G T: value(a;) G {0, 1}. 
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Fig. 2. SLTS and corresponding MTBDD 



ordering Si ^ ^ s„ ^ ^ ^ t„, the number of vertices is 3 • 2" — 1. 

Since identity matrices play an important role during the parallel composition 
of transition systems (see below), their compact representation is an essential 
feature of MTBDDs. 

A comprehensive set of logical and arithmetic operations can be realised 
efficiently on MTBDDs, such that it is possible to perform calculations on the 
functions which are represented by the decision diagrams. The operation Apply, 
for instance, combines two MTBDDs by a binary arithmetic operator. Restrict 
fixes the value of one or more variables of the MTBDD, and Abstract combines 
restricted copies of an MTBDD by an associative binary operator. In general, 
algorithms for MTBDD construction and manipulation are variants of their cor- 
responding BDD algorithms H21. They all follow a recursive descent scheme ac- 
cording to the above Shannon expansion, and their efficiency is achieved through 
the clever use of a hash-based vertex table and a cache where intermediate results 
are stored for later re-use mi- MTBDDs are very well suited for the compact 
representation of block-structured matrices, and symbolic algorithms for matrix 
multiplication and other linear algebra operations exist [ 4146151 1 . However, exist- 
ing implementations of MTBDD-based matrix multiplication and vector matrix 
multiplication are considerably slower than their sparse counterparts. 



4.2 Symbolic Representation of Transition Systems 

Fig. El shows an SLTS and its symbolic representation by an MTBDD. Since 
the set Act of this SLTS only contains two elements, a single Boolean variable 
suffices to encode the action label (the case a = 0 encodes action enq, and a = 1 
encodes action deq). Since the SLTS has four states, two bits are required to en- 
code the state identity. We use Boolean variables Si,S 2 to encoded a transition’s 
source state, and ti,t 2 to encode its target state. The transition S 2 ss, for 
example, is encoded by the combination (a, Si, ti, S2, t2) = ( 0 , 1 , 1 , 0 , 1 ). Note the 
interleaving of the variables for source and target state. 

If MTBDD M represents SLTS T we write M O T. For the symbolic rep- 
resentation of an ESTLS 7”, one employs two separate decision diagrams, i.e. 
an MTBDD which encodes all immediate transitions, and an MTBDD 
which encodes all Markovian transitions, as shown in the example of Fig. 0 We 
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Fig. 3. Encoding of an example ESLTS 

then write (M^, M^) l> T. Basically, is a BDD, since it does not encode any 
rate values. However, in certain situations one may wish to associate immediate 
transitions with numerical values, for instance to associate them with weights or 
probabilities, a feature which may be needed when resolving non-determinism be- 
tween several concurrently enabled internal immediate transitions. In this case, 
is a proper MTBDD with possibly more than two terminal vertices. Our 
tool Im-CaiO realises this scheme. A second alternative for the symbolic rep- 
resentation of ESLTSs, where both Markovian and immediate transitions are 
represented by a single MTBDD, is described in m 



4.3 Symbolic Manipulation and Analysis of Transition Systems 

In this section, we discuss the construction, manipulation and analysis of transi- 
tion systems represented by MTBDDs. Given a transition system, its symbolic 
representation can be easily constructed as the sum of the MTBDDs encoding 
the individual transitions. However, we recommend this procedure only for small 
transition systems, since the encoding of monolithical transition systems does 
usually not yield compact representations. Large transition systems should be 
generated in a compositional fashion from components, following the parallel 
composition operator of process algebras. 

Parallel composition: We now describe MTBDD-based parallel composi- 
tion, but for simplicity we restrict ourselves to the case of SLTS. The general 
ESLTS case works in a similar way. Consider the parallel composition of two 
SLTSs 7i and ?2 where actions from the set S C Act \ {r} shall take place 
in a synchronised way. Using process algebraic notation, we can express this as 
T = Til [S'] 1 72, where T is the resulting SLTS. Assume that the MTBDDs which 
correspond to SLTSs 7i and T 2 have already been generated and are denoted 
Ml and M 2 , i.e. M^ [>7i for i G {1,2}. The set of synchronising actions S can 
also be encoded in the standard way as a BDD, say S (action labels are encoded 
by the same Boolean variables in Mi, M 2 and S). The MTBDD M representing 



Im-Cat is a tool for the compositional construction and analysis of ESLTSs |44I45| 
which uses the CUDD library 
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T (i.e. M > T) is constructed as follows: 



M= (Mi-S)-(M2-S) 

+ Mi-(l-S)-ld2 + M2-(l-S)-ldi 

The term on the first line is for the synchronising actions in which both 7i 
and ?2 participate. The multiplication Mi-S selects that part of SLTS 7i which 
corresponds to actions from the set S, and similarly for M 2 -S. By then taking 
the product of these two terms one obtains the encoding of those transitions 
where both partners simultaneously make a move. The two symmetric terms on 
the second line are for those actions which 7i {T 2 ) performs independently of ?2 
(7i) — these actions are all from the complement of S, encoded by (1 — S) — and 
the multiplication with Id 2 (Idi) ensures that T 2 (7i) remains stable, i.e. does not 
change its state. Note that for the synchronising transitions, calculated by the 
first line in the above expression, the resulting rate is given by the product A • /i. 
Should one wish to employ a different function ^(A, /i), for instance the maximum 
function, one would simply have to replace the first line of the above expression 
by Max(Mi-S, M 2 -S), where Max is the maximum function on MTBDDs which 
can be realised with the help of a particular instance of the standard Apply 
algorithm. 

Enders et al. who considered the parallel composition of BDDs gen- 
erated from CCS terms, showed that the size of the symbolic representation is 
proportional to the sum of the sizes of its components, provided that the compo- 
nents are loosely coupled and provided that the interleaved variable ordering is 
used. We now state a similar result for the parallel composition of ESLTSs. Let 
71 and T 2 be two ESLTSs represented by MTBDDs, i.e. (Mf , l>% {i = 1, 2), 

using the interleaved variable ordering. Let (M^, M-^) > 7i|[5']|72 where is 
constructed from S and as above, and is constructed from M{, S 
and M 2 in a similar way. Then the number of vertices of is linear in the 
number of vertices of and M^, and the number of vertices of is linear 
in the number of vertices of M( and M^. For a proof of this important property 
see P2]. 

The fact that parallel composition of components can be realised symbolically 
in such a way that the size of the data structure grows only linearly compares 
favourably to the exponential growth resulting from the usual interleaving of 
causally independent transitions (as generated, for instance, by the operational 
semantics of process algebras) . This feature may be exploited in order to obtain 
extremely compact representations of huge transition systems. In fact, one can 
safely state that symbolic representations are only beneficial if they are used in 
a compositional context. 

Reachability analysis: The MTBDD M resulting from the parallel com- 
position of two partners Mi and M 2 encodes all transitions which are possible 
in the product space of the two partner processes (called the potential state 
space). Given a pair of initial states for SLTSs 7i and ? 2 , however, only part 
of this product space (the actual state space) may be reachable due to synchro- 
nisation constraints. Therefore, M potentially includes transitions emanating 
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from unreachable states. In this situation, reachability analysis is an important 
tool for reducing the size of the underlying SLTS. Reachability analysis can be 
performed efficiently on the symbolic representation of the resulting transition 
system T (as described for the purely functional LTS case in 0), both for SLTSs 
and ESLTSs represented by MTBDDs. 

Hiding: Hiding of action labels, i.e. replacing a visible action a by the in- 
ternal action r, can be performed on the MTBDD-based representation of an 
ESLTS with the help of the operations Restrict and Apply, by selecting and 
modifying that part of the MTBDD which encodes a-transitions, and by af- 
terwards recombining it with the remaining part of the MTBDD. While the 
hiding of Markovian transitions does not enable reductions of the transition sys- 
tem, the hiding of immediate transitions may lead to compositionally vanishing 
states which can be eliminated. In P!, we describe a symbolic algorithm for 
the elimination of such states that offers a flexible mechanism for resolving non- 
determinism between several internal immediate transitions, as realised in our 
tool Im-Cat. 

Bisimulation: Symbolic characterisation of strong and weak bisimulation 
and symbolic algorithms for computing a factorisation of the state space have 
been described in the literature jDfZ,4l4 ij . In PI, symbolic algorithms for com- 
puting strong and weak Markovian bisimulation on ESLTSs are described in 
detail, using decision node BDDs (DNBDD) m. another extension of BDDs 
for the representation of real- valued functions, as the underlying data structure. 
These algorithms follow the well-known strategy of iterative refinement and can 
readily be implemented with the help of MTBDDs. 

Numerical analysis: Numerical analysis can be carried out directly on 
the symbolic representation of the Markov chain msoEi]- Direct methods for 
calculating steady-state probabilities are generally unsuitable for symbolic im- 
plementation, since each step modifies the structure of the coefficient matrix 
and thus the MTBDD structure, which causes considerable overhead to keep 
the representation canonical and destroys its compactness For the analysis 
of large Markov chains based on their symbolic representation, iterative meth- 
ods are more suitable. Apart from a general matrix powering algorithm m that 
can be instantiated as the power method or the method of Jacob0, the projec- 
tion methods BiCGStab Wm and CGS m have been realised on the basis 
of MTBDDs. Unfortunately, the symbolic implementations of these algorithms 
are all substantially slower than their sparse-matrix counterparts, a fact which is 
due to the relatively poor performance of symbolic vector-matrix multiplication, 
as has been observed also in [l4l,4H;-i?SI44l/ 1) . 

4.4 Compactness of the Symbolic Representation 

As an example (taken from 1661 and also used in 1561 ). we consider a cyclic 
server polling system consisting of d stations and a server. The MTBDD rep- 

^ In principle, the method of Gauss-Seidel can also be realised by vector-matrix mul- 
tiplication, but this requires the inversion of a triangular matrix which usually leads 
to inefficient encodings. 
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Table 1. Statistics for the polling system 



d 


reach, states 


transitions 


MTBDD size MTBDD size 

compositional 

before reachability after reachability 


MTBDD size 
monolithic 


3 


36 


84 


169 


203 


351 


5 


240 


800 


387 


563 


1,888 


7 


1,344 


5,824 


624 


1,087 


9,056 


10 


15,360 


89,600 


1,163 


2,459 


69,580 


15 


737,280 


6.144e+6 


2,191 


6,317 


- 


20 


3.14573e-|-07 


3.40787e-|-08 


3,704 


13,135 


- 



resentation of the overall polling model (Tpoii) is constructed with the help 
of MTBDD-based parallel composition from d + 1 elementary transition sys- 
temfl one for the server {Tserv) and one for each station (Tstau), according to 
Tpoii ■= 7^e™|[S']|(7^tati|[0]| ■ • ■ |[0]|7^tatd)- The order in which the component 
MTBDDs are generated turned out to be of great importance for the resulting 
MTBDD size, since it determines the ordering of the MTBDD variables. Un- 
fortunately, it is not obvious a priori which component ordering yields small 
MTBDDs. 

In Tab.tn the sizes of the resulting MTBDDs are given for different values of 
d. The first column of the table contains the number of stations d, the 2nd (3rd) 
column contains the number of reachable states (the number of transitions), and 
the remaining columns give the number of vertices of the corresponding MTB- 
DDt0. Tab. □ shows that even for an extremely large state space, the MTBDD 
representation can be very compact, if it is constructed in a compositional fash- 
ion. The last column of Tab. Q shows the number of MTBDD vertices which one 
would obtain if one took the monolithic transition system of the overall model as 
generated by TIPPtool (which does not contain unreachable states), and di- 
rectly encoded it as an MTBDD. Clearly, this method cannot be recommended: 
Apart from the fact that the transition system of the overall model may not be 
available due to its excessive size and generation time (as indicated by the 
entries), the growth of the MTBDD sizes is prohibitive. As expected, the figures 
in column 4 grow linearly, whereas the ones in column 6 grow exponentially. 

The MTBDDs generated compositionally represent all transitions which are 
possible within the potential state space. As can be observed from the 5th col- 
umn of Tab.d determining the set of reachable states and “deleting” the transi- 
tions which originate in unreachable states considerably increases the size of the 
MTBDDs. Therefore, in general, although MTBDD-based reachability analysis 
is very fast, it is recommended to work with MTBDDs which represent the po- 
tential rather than the actual state space, and store the reachability predicate in 
a separate BDD. It may seem quite surprising that restriction to the reachable 
part of the transition system increases the size of the MTBDD. However, similar 

® The elementary transition systems were generated by the stochastic process algebra 
tool TIPPtool |23] and then encoded as individual MTBDDs. 

® Since the considered version of the polling model does not contain immediate tran- 
sitions, a single MTBDD (representing Markovian transitions) is sufficient. 



16 



M. Siegle 



phenomena can be observed when performing symbolic elimination of vanishing 
states or symbolic bisimulation: The size of the symbolic representation grows 
although the underlying transition system is reduced, i.e. fewer states and tran- 
sitions are represented. Our explanation for such counter-intuitive behaviour is 
that the regularity of the model gets lost through the reduction, which destroys 
the regularity of the MTBBD and thus its compactness (see m)- 

We now mention some figures concerning the MTBDD-based numerical anal- 
ysis of the polling system: For the case d = 7, and working on the potential state 
space, the MTBDD representing the power iteration matrix, as generated by IM- 
Cat, has 806 vertices and takes 0.8 seconds to construd0 One iteration of the 
power scheme takes at the average 0.122 seconds, but it takes a ridiculous 8070 
power iterations to converge. The MTBDD representing the Jacobi iteration ma- 
trix for the same system is larger, it has 1639 vertices and takes 18.94 second to 
construct, but one Jacobi iteration takes only 0.101 seconds and convergence is 
reached after 240 iterations. Unfortunately, these speeds are unacceptably slow, 
if compared to state-of-the-art sparse matrix implementations such as TIPP- 
tool’s solver (based on SparseLibl.3 by K. Kundert, UC Berkeley), where one 
Gauss-Seidel iteration takes only 0.0013 seconds. 



5 Discussion and Conclusion 

In this paper, we have reviewed the evolution from monolithic to modular model 
representations. In particular, we have described space-efficient symbolic rep- 
resentations of compositional Markov models stemming from process algebraic 
specifications, thereby emphasising the role of symbolic parallel composition. 

We briefly mention two other data structures related to decision diagrams 
and developed for the analysis of Markovian systems: Matrix diagrams I28I76I 
an extension of multi-valued decision diagrams, enable the compact rep- 
resentation of structured GSPN models, and probabilistic decision graphs m 
enable a consise representation of probability vectors and probabilistic transition 
system. 

As we have seen, the main bottleneck of the symbolic modelling procedure is 
numerical analysis. Therefore, speeding up MTBDD-based vector-matrix multi- 
plication remains a major area of research. A promising approach to this problem 
that combines the advantages of sparse and MTBDD-based representations will 
be described in isq. A totally different direction is taken in IZS], where special- 
purpose hardware for the support of basic MTBDD operations has been devel- 
oped. Parallelisation and distribution of MTBDD manipulation algorithms are 
also candidates for improving the speed of MTBDD-based numerical analysis. 

In summary, we argue that modular model specifications and symbolic repre- 
sentations are a perfect match, and that this combination should play a leading 
role in future performability analysis and verification projects. 

The experimental results were obtained on a SUN 5/10 workstation, equipped with 
1GB of main memory and running at 300 MHz. 
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Abstract. This paper reports on the implementation and the experi- 
ments with symbolic model checking of continuous-time Markov chains 
using multi-terminal binary decision diagrams (MTBDDs). Properties 
are expressed in Continuous Stochastic Logic (CSL) |Jj which includes 
the means to express both transient and steady-state performance mea- 
sures. We show that all CSL operators can be treated using standard 
operations on MTBDDs, thus allowing a rather straightforward imple- 
mentation of symbolic CSL model checking on existing MTBDD-based 
platforms such as the verifier PRISM. The main result of the paper is an 
improvement of 0{N) in the time complexity of checking time-bounded 
until- formulas, where N is the number of states in the CTMC under con- 
sideration. This result yields a drastic speed-up in the verification time 
of model checking CTMCs, both in the symbolic and non-symbolic case. 



1 Introduction 

In model-based performance and dependability evaluation, techniques such as 
stochastic Petri nets, stochastic process algebras, stochastic activity networks, 
and queueing networks are used to specify the system behaviour at a high level 
of abstraction. Most of these techniques assume a continuous-time Markov chain 
(CTMC) as underlying stochastic process. While the analysis of CTMCs focuses 
mostly on transient-state and steady-state (i.e. long run) characteristics, the 
specification and analysis of path measures is a subject of growing interest m 
The temporal logic CSL (Continuous Stochastic Logic) developed originally 
by Aziz et al. m and extended by Baier et al. [7] provides a powerful means 
to specify path-based as well as traditional state-based measures on CTMCs 
in a concise, flexible and unambiguous way. CSL is based on the well-known 
branching-time temporal logic CTL (Computation Tree Logic ^J) and PCTL 
(Probabilistic CTL [17|'l: a steady-state operator, a time-bounded until, and a 
probabilistic (path) operator constitute its main ingredients. It allows one to 
state, for example, that the probability of reaching a certain set of goal-states 
within a specified real- valued time bound, provided that all paths to these states 
obey certain properties, is at least/at most some probability value. 

* Partly supported by EPSRC grants GR/M04617, GR/M13046 and GR/N31573. 
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Verification of a given finite-state CTMC against a CSL formula is performed 
using model checking. The model checking problem for CSL is decidable for ra- 
tional time bounds m- Approximate CSL model-checking algorithms have been 
studied in [ 7 | where the satisfaction of time-bounded until formulas is shown to 
be based on solving a (recursive) Volterra equation system. More recently, Baier 
et al. reduced verifying time-bounded until formulas to the problem of com- 
puting transient-state probabilities for CTMCs. This significant result employs 
a formula-dependent transformation of the CTMC and - more importantly - 
allows one to adopt efficient techniques like uniformisation |TfTF^ for verifying 
time-bounded until- formulas. This paper builds upon this earlier work, and con- 
siders two issues: improving the time and the space efficiency of model checking 
CTMCs against CSL formulas based on transient analysis. 

Faster CSL model checking. Verifying time-bounded until formulas using tran- 
sient analysis of CTMCs suggests that uniformisation should be applied to 
each individual state separately. This results in a worst case time complexity of 
0{M-N), where N is the number of states in the CTMC and M the number of 
transitions. The main result of this paper is an improvement of 0{N) in the time 
complexity of checking time-bounded until formulas. Inspired by PCTL model 
checking, the basic idea underlying this efficiency improvement is to carry out 
the uniformisation for all states at once. Our experiments show that this result 
yields a drastic speed-up in the verification time of model checking CTMCs. 

Symbolic CSL model checking. To combat the infamous state-space explosion 
problem we investigate representing the state space by multi-terminal binary 
decision diagrams (MTBDDs ^2], also called algebraic decision diagrams 0). 
MTBDDs are variants of BDDs that can efficiently deal with real matrices; they 
allow arbitrary real numbers in the terminal nodes instead of just 0 and 1. We 
show that CSL model checking can be treated using standard operations on 
MTBDDs, thus generalising the result for PCTL |t>l1 SIJ to the continuous-time 
setting. This basically follows from the fact that CSL model checking amounts to 
the analysis of either the embedded discrete-time Markov chain (DTMC) - in the 
case of untimed until formulas and steady-state formulas - or the uniformised 
DTMC - in the case of time-bounded until - of the CTMC under consideration. 
This reduces to graph analysis and iterative matrix-vector multiplication which 
can be implemented with standard MTBDD operations. Variants of MTBDDs 
tailored to numerical integration [[j are not needed. This paper reports on the 
implementation of symbolic CSL model checking as part of PRISaQ (PRoba- 
bilistlc Symbolic Model checker), a prototype tool for the symbolic verification 
of Markov decision processes (MDPs) with DTMCs as a subset thereof. 

Organisation of the paper. Section El briefly recalls PCTL model checking. Sec- 
tion 0 introduces CTMCs and CSL. Handling time-bounded until is covered in 
Section 0 Section 0 discusses CSL model checking with MTBDDs. Section 0 
presents our empirical results. Section 0 concludes the paper. 

WWW. cs .bham. ac .uk/~dxp/prism 
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2 The Discrete-Time Setting 



DTMCs. Let AP be a fixed, finite set of atomic propositions. A (labelled) DTMC 
2? is a tuple (S,P,L) where S' is a finite set of states, P : S x S — >■ [0, 1] is a 
probability matrix such that X^s'gS ^ s G S, and L : S ^ 2^^ 

is a labelling function which assigns to each state s G S the set L(s) of atomic 
propositions that are valid in s. A path through a DTMC is a sequence0 of states 
a = So Si S 2 . . . with P(si, s^+i) > 0 for all i. Let PatlP denote the set of all 
paths in V. a[i] denotes the (i+l)th state of a, i.e. cj[i\ = s^+i. Let Prg denote 
the unique probability measure on sets of paths that start in state s m- 



PCTL. Let a G AP, p G [0, 1], A: be a natural (or oo) and to G { ^, ^ }• The 
syntax of PCTL is: 



<l> ::= tt 



a 



(p A<P 



~^<P 



V^p{<PU^^ <P) 



The other boolean connectives are derived in the usual way. For the sake of 
simplicity, we do not consider the next state operator in this paper. The standard 
(i.e. unbounded) until formula is obtained by taking k equal to oo, i.e. <PUW — 
W. The semantics of PCTL is defined by |17| : 



s 1= tt for all s G S' 
s 1= a iff a G L{s) 
s 1= iff s ^ 



s\=(p A ^ iffs^^As)=iF 

iff Pro&^(s,<l>W^'= IF) top 



Pmp(^A 2^^ !F) asserts that the probability measure of the paths that start in s 
and that satisfy 'P meets the bound to p. Here, 



Prob'^{s,PU^^P) = Pr^cr G Path^ \ a h 



Formula <PU^^ P asserts that P will be satisfied within k steps and that all 
preceding states satisfy P, i.e.: 



cr 1= P iff 3j ^ k. (cr[j] \= P A'ii < j. a[i] |= P) 



Model checking PCTL. PCTL model checking im is carried out in the same way 
as verifying CTL m by recursively computing the set Sat{P) = { s G S | s ^ 
P}. Checking bounded until formulas amounts to computing the least solution 
of the following set of equations: Prob^ {s,PhT^^ P) equals 1 if s G Sat(P), 

Prob'^{s,PU^^P) = P{s,s')-Prob'^{s',PU^^-^P) (1) 

s'es 

if s G Sat{P A -'P) and fc > 0, and equals 0 otherwise. For DTMC V = {S,V,L) 
and PCTL formula P, let DTMC = {S,P',L) where if s P, then 
P'(s, s') = P(s, s') for all s' G S, and if s ^ then P'(s, s) = 1 and P'(s, s') = 0 

^ In this paper, we do not dwell upon distinguishing finite and infinite paths. 
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for all s' 7 ^ s. We have = T>['P V 'F]. Let {s, k){s') denote the prob- 

ability of being in state s' after k steps in DTMC T> when starting in s, i.e. 
7T^(s, k){s') = Prg{ cr S Path^ \ a[k] = s' }. 

Proposition!. For DTMC V: Pro}P F) = ^ fc)(s'). 

s'|='i' 

Note that D[-^<F\/F] = D[-^{F /\F)]\F], i.e. all -'(^A'f')-states and all if'-states in 
D are made absorbinJE The former is correct since FU^^F is violated as soon 
as some state is visited that neither satisfies F nor F. The latter is correct since, 
once a !f'-state in T> has been reached (along a <?-path) in at most k steps, then 
FU^^ F holds, regardless of which states will be visited later on. 

Model checking for all states thus amounts to computing 

where characterises Sat(F), i.e. i^(s) = 1 if s |= tZ/, and 0 oth- 
erwise. As iterative squaring is not attractive for stochastic matrices due to fill 
in the product is typically computed in an iterative fashion: P.(...(P-6^))). 



3 The Continuous-Time Setting 



CTMCs. A (labelled) CTMC C is a tuple (S', R, L) where S and L are as for 
DTMCs, and R : S x S — )> is the rate matrix. (We adopt the same conven- 
tions as in E|71,i.e. we do allow self-loops.) The exit rate E{s) = J2s'eS ■^0 
denotes that the probability of taking a transition from s within t time units 
equals If R( s, s') > 0 for more than one state s' , a race between the 

outgoing transitions from s exists. That is, the probability P(s, s') of moving 
from s to s' in a single step equals the probability that the delay of going from 
s to s' “finishes before” the delays of any other outgoing transition from s. 

Definition 1. For CTMC C = (S, R, L), the embedded DTMC is given by 
emb{C) = (S, P, L), where P(s, s') = R(s, s')/E(s) if E{s) > 0, and P(s, s) = 1 
and P(s, s') = 0 for s s' if E{s) = 0. 

A path through a CTMC is an alternating sequence a = sq to si h S 2 . ■ . with 
R(si, Si+i) > 0 and U S IR>o for all i. The time stamps U denote the amount of 
time spent in state Si. Let Patif denote the set of paths through C. a@t denotes 
the state of cr occupied at time t, i.e. a@t = a\i] with i the smallest index such 
that t ^ denote the unique probability measure on sets of paths 

that start in s jZ]. 



CSL. Let a,p and to be as before and t G M^o (or oo). The syntax of CSL is: 



F ::= tt 



a 



F AF 



~^F 



'5[xip(<?) 



V^p{FU^' F) 



S^p{F) asserts that the steady-state probability for a ^-state meets the bound 
CXI p. The semantics of CSL for the boolean operators is identical to that for 

® That is, the only transitions available in these states are self-loops. 
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PCTL. For the remaining state formulas 

s \= iff linit_>.oo Prs{ c G Path‘S \ a@t \= <P} t<ip 

s\=V^p{^U^*^) iS Prob‘d 

The limit in the first equation always exists as C contains finitely many 
states m Prob‘d {■) is defined in a similar way as for PCTL: 

Prob^ <P) = Prs{ cr G Patlf \ a [= . 

The operator is the real-time variant of the PCTL operator 

asserts that <F will be satisfied at some time instant in the interval [0, t] and that 

at all preceding time instants <P holds: 

a \= iff 3cc < t. (a@x \= ^ A\/y < x. a@y \= <P) . 

Note that the standard until operator is obtained by taking t equal to oo. 

CSL model checking m is performed in the same way as for CTL nn and 
PCTL jIZ|, by recursively computing the set Sat{<!>). For the boolean operators 
this is exactly as for CTL and for unbounded until this is exactly as for PCTL. 

Model checking the S operator. For determining Sat{S,^p{’P)), first Sat{<P) is 
computed (as usual), and a graph analysis is carried out to determine the bottom 
strongly connected components (BSCCs) of C, i.e. the set of SCCs in C that, 
once entered, cannot be left any more. The steady-state probability distribution 
7T® inside each BSCC B is determined using standard means EH]: by solving a 
linear equation system in the size of the BSCC at hand. Then, the probabilities 
of reaching a BSCC B from a given state s are computed for each B. State s 
now satisfies if: 



Pr{ reach B from s} • ixip 

B y s'GSnSat(<*>) / 

All these steps can be performed on the embedded DTMC as timing issues are 
not involved; for details see [Zj. 

Model checking the operator. Checking time-bounded until formulas is based 
on determining the least solution of the following set of integral equations: 
Prob^ {s,<PU^* equals 1 if s G Sat(\P), 

Prob^{s,<PU^*W) = f ■ Prob^{s',$U^^-^W)dx 

•^0 s'gS 

if s G Sat{<P A and equals 0 otherwise. Here, denotes the 

probability density of taking some outgoing transition from s at time x. Note 
the resemblance with equation jn) for the PCTL bounded until operator. For 
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CTMC C = {S,R,L) and CSL formula let CTMC C[^>] = {S,R',L) with 
R'(s, s') = R(s, s') if s ^ and 0 otherwise. Note that emb(C[^]) = emb(C)[<?]. 
It has been shown in that for a given CTMC C and state s in C, the mea- 
sure Prob^ can be calculated by means of a transient analysis of 
the CTMC C', which can easily be derived from C using the [•] operator. Let 
7t‘"(s, f)(s') denote the probability of being in state s' at time t given that the 
system started in state s, i.e. 7t^(s, t)(s') = Pr^j a G Patif \ <7@t = s' }. 

Theorem 1. For CTMC C: Prob^ ^) = ^ 7r'^[^'^''®'l(s, t)(s'). 



4 Faster Time-Bounded until Verification 

In this section, we present an algorithm for verifying time-bounded until formulas 
that is based on (i) the aforementioned reduction to transient analysis and on 
(ii) the algorithm for PCTL bounded until. This combination - suggested by the 
strong resemblance of Theorem Hand Proposition yields an improvement of 
0{N) in time complexity over the algorithm suggested in where N is the 
number of states in the CTMC. We first briefly describe uniformisation. 

Uniformisation. Uniformisation is a transformation of a CTMC into a DTMC: 

Definition 2. For CTMC C = {8,11, L) the uniformised DTMC is given by 
unif{C) — {S,P,L) where P = I -|- Q/g for q ^ max{U(s) | s G S'} and Q = 
R — diag(E) . 

The uniformisation rate q is determined by the state with the shortest mean 
residence time. All (exponential) delays in the CTMC C are normalised with 
respect to q. That is, for each state s G S with E{s) = q, one epoch in unif(C) 
corresponds to a single exponentially distributed delay with rate q, after which 
one of its successor states is selected probabilistically. As a result, such states 
have no self-loop in the DTMC. If E(s) < q - this state has on average a longer 
state residence time than ^ - one epoch in unif{C) might not be “long enough”. 
Hence, in the next epoch these states might be revisited and, accordingly, are 
equipped with a self-loop with probability 1 — . Note the difference between 

the embedded DTMC emb{C) and the uniformised DTMC unif{C): whereas the 
epochs in C and emb{C) coincide and emb{C) can be considered as the time- 
less variant of C, a single epoch in unif{C) corresponds to a single exponentially 
distributed delay with rate g in C. 

Transient analysis. The probabilities {s,t){s') are now computed as follows: 

7r(s,t) = 7 t(s,0) • = ^l{k,q-t) ■ 7r(s,fc) (2) 

k=0 ■ k=0 

where P is the probability matrix of the DTMC unif{C), and ■j{k,q-t) is the 
fcth Poisson probability with parameter q-t, i.e. ^{k,q-t) = e~^'^-{q-f)^ /k\. The 
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vector 7t(s, fc) denotes the probability distribution in unif(C) after k epochs when 
starting in s, i.e. 7r(s, fc) = 7r(s, 0) • P^, where 7r(s, 0)(s) = 1 and 7r(s,0)(s') = 0 
if s yf s'. Equation © can be understood as follows. During the time interval 
[0, t), with probability j{k, q-t) exactly k jumps have taken place in the DTMC 
unif(C). The effect of these jumps is described by 7t(s,0)-P^. Weighting this 
vector with ■j(k,q-t) and summing over all possible numbers of jumps in 
we obtain, by the law of total probability, the probability vector n^{s,t). 

Given an accuracy e, the number of terms Re of the infinite summation in 
m that have to be considered is the smallest value satisfying: 



{q-tr ^ 1 - e 

n=0 






For large q-t, is of order 0{q-t)^ As the first group of Poisson probabilities 
are typically very small, the first terms in 0 are negligible and need not be 
computed. and Re are called the left and right truncation point, respectively. 



A first algorithm. The algorithm for time-bounded until as suggested in |0j is 
based on carrying out a computation according to equation Q and the fact that 
7 t(s, k) = 7t(s, 0) • P^. The computation is carried out in an iterative manner per 
individual state s starting from the initial distribution 7r(s,0). The pseudo-code 
of this algorithm is presented in Fig. [0 Here, and in the subsequent algorithms 
in this paper, the Poisson probabilities are computed using the Fox-Glynn algo- 
rithm uni that avoids overflow for large q-t. The overall time complexity of this 
procedure is O(N-q-t-M), where q is the uniformisation rate of the GTMG at 
hand, t the time bound of the until formula, N the number of states and M the 
number of non-zero entries in R. This follows directly from the fact that for each 
state the number of terms of @ that needs to be considered is 0{q-t), where 
each term requires a matrix vector multiplication with 0{M) multiplications 
given a sparse data structure. 



An alternative algorithm. The basic idea of the new algorithm is to use the 
iterative matrix vector multiplication of the PGTL bounded until operator as a 
basis, and impose the computation of the Poisson probabilities on top of it. This 
is suggested by the following observation: 

OO 

Proposition 2. Prob‘d {s,<PU^*<R) = 

k=0 

Recall that 'y{k,q-f) denotes the probability of taking k jumps in the DTMG 
unif{C) in the interval [0, t). From Propositions ^ and |2| it follows that the vector 
Prolf can be obtained in an iterative manner, cf. the pseudo-code in 

Fig.El As a result, a global transient analysis is carried out, yielding for each 
state s the probability measure Prob^{s, ^). Note that, as opposed to the 

Note that the DTMC unif(C) may reach steady state before Re and, in this case, 
the summation can be truncated at this earlier point m- 
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// compute Poisson probabilities 
7, Le, Re FoxGlynn(5 • t, e) 

/ / main loop 
foreach s £ S 
sol := 0 
p := 7t(s,0) 
for fc = 1 to Le — 1 
p ;= p • P 
endf or 

for k = Le to Re 
p ;= p • P 

sol := sol + 'y{k, q-t) ■ p 
endf or 

// Prob{s, ^) = sd-il 

endf or 



Fig. 1. A first algorithm 



// compute Poisson probabilities 
7, Le, Re ~ FoxGlynn((j • t, e) 
// main loop 
sol 0 

b_ ■— ijp 

for fe = 1 to Le — 1 
b--P-h 
endf or 

for fc = Le to Re 
b-.= P-b 

sol := spZ + y{k, q-t) ■ b 
endf or 

// Proh {<PU^*m =sd 



Fig. 2. An efficient variant 



algorithm of Fig. ^ is not a probability vector in Fig. 0, i.e. its elements 
do not sum up to one. It is evident from the efficiency considerations given just 
before, that the time complexity of the adapted algorithm is 0{q-t-M), thus 
yielding an improvement of 0{N) over the previous algorithm. 

5 Symbolic Model Checking CTMCs with PRISM 

Due to the recent improvements in verification time - including our suggested im- 
provement - space efficiency considerations become more important for CTMC 
model checking. In this section, we report on symbolic model checking of CTMCs 
(against CSL) using MTBDDs. MTBDDs have the ability to exploit structure 
(regularity) in models and can represent them in a far more compact way than 
a sparse matrix would. The success of BDD-based model checking in the non- 
probabilistic case serves as sufficient motivation to develop the foundations of 
MTBDD-based model checking and experiment with these techniques. 

MTBDDs. Let xi < X 2 < ■ ■ ■ < Xn distinct, totally ordered state variables. 
An MTBDD over (xi, . . . ,Xn) is a rooted directed graph with vertex set V 
containing two types of vertices: 

— each non-terminal vertex v is labelled by a state variable var{v) £ { Xi, . . . , 
Xn } and has two children left{v), right (v) £ V 

— each terminal vertex v is labelled by a real number val(v), 

such that var{v) < var{w) for each non-terminal vertex v and non-terminal child 
w of v. The constraint requires that on any path from the root to a terminal ver- 
tex, the variables respect the ordering <. An MTBDD M over (xi, . . . , a;„) repre- 
sents the function /m : { 0, 1 }" — >■ K, whose values are obtained by traversing M 
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starting at the root vertex as follows. For non-terminal vertex v, the edge from 
V to left{v) represents the case when var(v) is false; the edge from v to right(v) 
the case var{v) is true. For efficiency reasons, MTBDDs are usually stored in a 
reduced form ^D|. Note that a BDD is an MTBDD with val{v) € { 0, 1 } for all 
terminal vertices v. 

Representing CTMCs by MTBDDs. Let C = {S, R, L) be a CTMC with [S'! = 2" 
and L injective. (Any labelled CTMC may be transformed into one satisfying 
these conditions by adding dummy states and new propositions.) Let oi, . . . , a„ 
be an enumeration of the atomic propositions and identify each state s with 
the boolean n-tuple (5i,...,6„) where 6^ = 1 iff S L{s). This encoding 
of states is standard mm- Thus, S = { 0, 1 }" where each state s is iden- 
tified with its encoding and R with the function F : { 0, 1 }^" — >■ IR where 
F{xi,yi, . . .,Xn,yn) = R((a;i, . . .,Xn), {yi, ■ ■ ■ ,2/n))- 

Operations on MTBDDs. Model checking CTMCs can be performed with stan- 
dard operations on MTBDDs. For completeness, we briefly describe these here. 
The operator Apply allows a point-wise application of the binary operator op 
(e.g. -I- or x) to two MTBDDs. For MTBDDs Mi and M 2 , Apply(op, Mi, M 2 ) 
yields an MTBDD for function /m^ op/ivij. For MTBDDs R and b represent- 
ing matrix R and vector b respectively, MTBDD MVMult(R, b) represents 
the vector R • b. For g G M, Const((j) denotes the MTBDD consisting of a 
single terminal vertex v with val{v) = q. For an MTBDD M, FindMax(M) 
returns the maximum value of the terminal vertices of M. The COMP operator 
takes an MTBDD M and an interval /CM and returns the BDD represent- 
ing the function that equals 1 if fuixi, . . . ,Xn) G I and 0 otherwise. Operator 
Abstract (op, M, xi, . . . , a^n) returns the MTBDD which results from abstract- 
ing all of the variables a;i, . . . , from M by applying op over all possible values 
taken by these variables. 

MTBDD-based model checking of CSL. The symbolic model checking algorithm 
for CSL is identical to the one proposed in except that we use transient analy- 
sis and uniformisation rather than numerical integration (which needs dedicated 
variants of MTBDDs). Let C = (S', R, L) be a CTMC represented by MTBDD 
R as explained above. For each CSL formula a BDD Sat|^] is defined that 
represents the characteristic function of the set Sat{T>). By applying standard 
operators on MTBDDs we determine the MTBDDs P representing the transition 
probability matrix P of emb(C), and E the vector of exit rates E_. Then: 

Sat|tt] = Const(I) 

Sat|ai ] = the BDD for the boolean function (xi, . . . , x„) >->■ Xi 
Sat|^/>] = NOT(Sat|/>l) 

Sat|^ A >/'] = AND(Sat|</], Sat|!F]) 

Sat|Sxip(^)] = COMP(STEADYSTATE(P,Sat|^]),[Xlp) 
Sat|P>4p(^Wtf')] = COMP(UNTiL(P,Sat|^],Sat|tf']),[xip) 

Sat|P^p(^W^‘ S') ] = Comp(TBUntil(R, E, Sat|^], Sat|/^J, t, e), ixi p). 
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algorithm TBUntil(R, E, SatJ^J, Sat| !f"], t, e) 

// uniformisation 

R' — Apply(x, NoT(OR(Nox(Sat|^]), Sat|'?'])), R) 

E' ;= Apply(x, NoT(OR(Nox(Sat|^]), Sat|!f'])), E) 
q ~ FindMax(E') 

Q := Apply(— , R', Apply( x , E', Identity)) 

P := Apply(+, I, Apply(^, Q, Consx(5))) 

// compute Poisson probabilities 
7, Le,Re FoxGLYNN(g • t, e) 

// main loop 
sol CONSx(O) 
b Sat|?F] 
for k = 1 to Le — 1 
b — MVMuLx(P,b) 
endf or 

for k = Le to Re 

b — MVMuLx(P,b) 

sol Apply(+, sol, Apply(x , CONSx(7(fe, q-t)), b)) 
endf or 
return sol 
end. 



Fig. 3. MTBDD algorithm for CSL time-bounded until using transient analysis 



Here, Sat|ai] is a HDD consisting of a single state vertex v labelled with Xi 
such that left{v) and right{v) are labelled with 0 and 1, respectively. The steady 
state and unbounded until operators are treated symbolically as described in jS] 
and jZj respectively. 

TBUntil assigns to each state s G S the probability (with precision e) of 
the set of paths that start in s fulfilling i.e. it represents the function 

s I— Prob{s,<PU^* 'P). The algorithm for TBUntil is shown in Fig. |3 where 
Identity denotes the MTBDD representing an identity matrix of the appropriate 
size. In the first two lines, the CTMC C[-><P V 'P] is computed. Note that the 
Apply operator filters out the states that become absorbing, i.e. the states that 
do not satisfy V <F). In the subsequent three lines, the uniformisation rate 

q and the DTMC unif(C) are determined. The rest of the pseudo-code is the 
MTBDD-based counterpart of the algorithm shown earlier in Fig. 0 

PRISM. PRISM is a verifier for discrete probabilistic systems such as DTMCs 
(against PCTL) and MDPs (against PCTL |2| with fairness |B|). The tool is 
implemented using a combination of Java and C-P-P. The high level parts of 
the tool, such as the user interface and the parsers, are written in Java. The 
engines and libraries are mostly written in C-P-P. PRISM takes as input a model 
description in a probabilistic variant of reactive modules 0 , constructs the model 
from its description and computes the set of reachable states. Model checking 
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Fig. 4. The PRISM tool architecture 



using different data structures is supported, cf. Fig. 0 symbolic representations 
using (MT)BDDs, conventional sparse matrices, and a hybrid approach using 
MTBDDs for storing matrices and conventional representations for probability 
vectors. For the manipulation of the symbolic data structures, PRISM uses the 
CUDD package [221 which is written in C. More information about the tool can 
be found at www.cs.bhami.ac.uk/~dxp/prism. 

The MTBDD based model checking algorithm for CSL has been implemented 
in PRISM, thus extending its applicability to continuous probabilistic systems. 
The realisation in PRISM includes an “on-the-ffy” steady state detection as part 
of the transient analysis (as in unj). For the sake of clarity, this mechanism is 
not included in the algorithm in Fig.0 

6 Experiments 

The case studies. To facilitate a comparison with E I— MC' ■ m, we consider two 
case studies that have been verified previously with CSL: a tandem network m 
and a cyclic server polling system EH- 

The tandem network consists of a M/C 0 X 2 / 1-queue and a M/M/ 1-queue, 
both of capacity K, put in sequence. Jobs arrive at the first station with rate 4-K. 
The first server executes jobs in either done or two phases, i.e. with probability 
0.9 a job is served once (with rate 2), and with probability 0.1 the job has to 
pass an additional phase (with rate 2). Once served by the first station, jobs are 
queued in the second station where service takes place with rate 4. The properties 
we verify for this model are the following probabilistic path properties: 

— full, i.e. the tandem network becomes fully occupied within t time units 

— fst, i.e. the first station of the tandem network becomes fully occupied 
within t time units 



where <P = tt <P. 

The polling server polls K stations in a cyclic fashion. The times for 
generating a message, for polling a station and for serving a job by a station 
are all distributed exponentially with rates 1/K, 200 and 1, respectively. If the 
server finds a station idle, then the service time is zero. For this system, we check 
the property busyi => 'P^p{0^*polli), i.e. once the first station has a job to be 
served it will be polled within t time units with probability co p. 
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Statistics and assessment. We ran all experiments on a 440 MHz SUN Ultra 
10 workstation with 512 Mb memory under the Solaris 2.7 operating system. 
All properties were checked with an accuracy e = 10“®. The verifiers PRISM 
and El— MC^ provide the results for the symbolic and sparse implementations 
respectively. 



Table 1. Comparison of the original and improved sparse implementations of time- 
bounded until algorithm 



sparse matrix implementation 


model 


K 


if states 


formula 


time (in sec) 


original 


improved 


tandem 

network 


2 


15 


V^plP^'^full) 


0.07 


0.01 


Pmp(O^"7u/0 


0.13 


0.01 




0.71 


0.03 




1.29 


0.09 


20 


861 


V^piO^'^full) 


563.94 


0.55 


Pxp(0^"7u//) 


1927.59 


1.01 




1978.22 


1.05 




1954.01 


1.00 


polling 

system 


3 


36 


busy^ ^ ■Pixp(O^^pollj) 


0.96 


0.02 


5 


240 


busy^ ^ Pt^p{0^"‘poll^) 


59.55 


0.16 


7 


1,344 


busy^ ^ Vixip{0^"‘pollj^) 


2637.11 


1.71 


10 


15,360 


busy^ ^ Pc^p{0^^pollj^) 


- 


13.48 



Statistics and assessment for the improved method. The statistics in Table E 
compare the verification times of the original sparse implementation of time- 
bounded until (Fig. nj and the improved algorithm (Fig. EJ. As expected, the 
results confirm that the improved algorithm is a factor of N faster, where N is 
the size of the state space. We note that, in several cases, there is an even greater 
speed-up. This is possibly due to the different computation steps performed by 
the two algorithms: the original works via a forwards exploration of the state 
space, whereas the improved version works backwards. To see this, note the 
difference between the iteration steps p := p • P in the original as opposed to 
6 := P • 6 in the improved. 

Statistics and assessment for the symbolic implementation. We now compare our 
symbolic implementation of time-bounded until with its sparse counterpart. For 
the tandem network and polling system examples, we have constructed efficient 
MTBDD representations of the transition matrix using the methods presented 
in PI (for further details see www . cs . bham . ac . uk/'dxp/prism) . This allows us 
to build and store much larger models with MTBDDs (given regularity) than is 
feasible with a sparse implementation. 
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Table 2. Comparison of symbolic and sparse verification of the tandem network 



sparse versus symbolic implementation 



K 


# states 


time per iteration (in sec) 






symbolic 


sparse 


symbolic 


sparse 


63 


8,128 


0.08 


0.01 


0.02 


0.01 


127 


32,640 


0.17 


0.04 


0.04 


0.05 


255 


130,816 


0.37 


0.55 


0.06 


0.15 


511 


523,776 


0.81 


1.50 


0.10 


0.71 


1023 


2,096,128 


- 


- 


0.23 


- 


2047 


8,386,560 


- 


- 


0.31 


- 


4095 


33,550,336 


- 


- 


0.66 


- 




Fig. 5. Comparison of symbolic and sparse verification of both examples 



The results of the comparison of our symbolic implementation with the sparse 
implementation are presented in Table |2|and Fig. 0, We have measured time per 
iteration as both implementations follow the improved algorithm given in Fig.|2| 
Table El summarises the results for the tandem network example based on two 
CSL properties. Fig.|5|gives time per iteration plotted against the size of the state 
space for both the tandem network and polling system for the CSL properties 
'Pc^p{0^^full) and busyi respectively. 

The efficiency of the symbolic time-bounded until implementation depends 
on the size of the MTBDDs representing the iteration vectors (5 and scd in 
Fig. El- Our experiments show that these are usually significantly larger than 
the MTBDD for the transition matrix, because of their relative lack of structure. 
For vectors to be represented compactly by MTBDDs the main requirement is 
a limited number of distinct elements. This condition is dependent on both 
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the structure of the model and on the property being verified, and as such it 
is difficult to determine when these vectors will be represented compactly. For 
example, compare the difference in performance of the symbolic implementation 
on two different models, as shown in Fig. 0 The times for the tandem network 
are much faster than for the polling system for models of equivalent size. 

On the other hand, in the sparse implementation the time complexity is 
dependent purely on the number of non-zeros in the matrix used for the com- 
putation. In Fig. |S( the times for the sparse approach can be seen to be almost 
identical for the tandem network and the polling system (note that in both ex- 
amples the number of non-zeros in the rate matrix is linear in the size of the 
state space). 

Comparing the results for the two implementations confirms, as expected, 
that we can verify larger models using a symbolic as opposed to a sparse ap- 
proach; for example, we were able to verify systems with 33 million states. A 
more surprising observation which we note for the first time is that, for certain 
models and certain properties, symbolic analysis is faster than sparse. So far, 
see e.g. PI, the sparse implementation has always outperformed the MTBDDs 
on quantitative numerical calculations. 

We are currently extending PRISM to improve the efficiency further by taking 
a hybrid approach which uses an MTBDD representation for storing matrices 
and a conventional representation for probability vectors. Early experiments 
show that, although slower than a sparse implementation, it is significantly faster 
than the pure MTBDD version. Like sparse, its performance is independent of 
the regularity of the model being considered, but it retains the advantage of 
MTBDDs, in that larger models can be represented. More information will be 
available in pH). 

7 Concluding Remarks 

This paper considered both space and time efficiency issues of CSL model check- 
ing of CTMCs. We presented an improvement in time efficiency of 0{N) for 
verifying time-bounded until formulas. The obtained empirical results indicate a 
drastic improvement in run times, making model checking of systems of realistic 
size feasible. In addition, we reported on symbolic model checking of CSL using 
MTBDD based uniformisation and transient analysis. Although, for simplicity, 
we have restricted the exposition in this paper to an until operator with time 
bounds [0,t], the results of our paper carry over to for arbitrary interval 
/ C IR;^o in n straightforward manner. 
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Abstract. We report on a novel development to model check quantita- 
tive reachability properties on Markov decision processes together with 
its prototype implementation. The innovation of the technique is that 
the analysis is performed on an abstraction of the model under analy- 
sis. Such an abstraction is significantly smaller than the original model 
and may safely refute or accept the required property. Otherwise, the 
abstraction is refined and the process repeated. As the numerical anal- 
ysis necessary to determine the validity of the property is more costly 
than the refinement process, the technique profits from applying such 
numerical analysis on smaller state spaces. 



1 Introduction 



The verification of systems has nowadays reached a clear maturity. Fully auto- 
matic tools, in particular model checkers, have been developed and successfully 
used in industrial cases. A model checker is a tool that can answer whether the 
system under study satisfies some required property. Many times, however, these 
type of properties are not expressive enough to assert adequately the correct- 
ness of a system. Nevertheless, it is desirable that the probability of reaching the 
unavoidable error is small enough. Quantitative model checking, that is, model 
checking of probabilistic models with respect to probabilistic properties, has al- 
ready been studied during the last decade |ldl21bl2lTEl etc.]. However, it was not 
until recently that attention was drawn to efficient tool implementations. In this 
paper we report on a novel development to model check quantitative properties. 

We use Markov decision processes (see e.g. m) to describe the system under 
study. This model, also called probabilistic transition system (PTS), allows to 
combine probabilistic and non-deterministic steps and is a natural extension to 
traditional non-deterministic models (such as labelled transition systems). Our 
preference for a probabilistic model that allows non-determinism is based on two 
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facts. First, PTSs are closed under parallel compostition which facilitates the 
modelling process. Second, PTSs are also closed under abstraction. This reason 
is fundamental as the method introduced in this paper is based on abstraction 
techniques. 

We focus on a restricted set of reachability properties. They allow to spec- 
ify that the probability to reach a particular final condition f from any state 
satisfying a given initial condition i is smaller (or greater) than a probability p. 
This type of properties is not so restrictive as it seems since we can always use 
checking automata to add additional constraints to the property. 

The method we present is based on automatic abstraction and refinement 
techniques. The basic idea is to use abstraction to reduce the high cost of proba- 
bilistic analysis. The difficulty lies in finding the right abstraction level, depend- 
ing on the property to prove. To address it, the method starts with a coarse 
abstraction of the system which is obtained by partitioning the state space, 
according to the property under study. The property is then checked on the ob- 
tained abstract model. The verdict may be inconclusive, that is, p happens to be 
between the calculated upper bound of the minimum and the lower bound of the 
maximum actual probabilities. In this case the previous abstraction is refined 
and the question posed again. The process is successively repeated until a satis- 
factory answer is given, or no further refinement is possible. To efficiently store 
the state space, perform abstractions and process the refinement steps, we use 
Bdds and Mtbdds (more precisely Adds) |1 ( )I,S| . The soundness of the method 
is asserted by considering a suitable probabilistic simulation (which pre- 

serves the kind of property we consider), and by showing that abstraction by 
partitioning respects this simulation relation. 

The contributions of this paper are first the definition of the probabilistic 
simulation relation that allows to prove the soundness of our method, and sec- 
ondly, the design of efficient algorithms to abstract PTSs, to analyse and to 
refine them. Finally, experimental results shows the effectiveness of the method. 



Related work. The partition refinement method we use on PTSs resorts to princi- 
ples already applied to finite-state systems jSI and timed automata • However 
our aim is not to generate a minimal model w.r.t. a bisimulation relation, but 
to steer the refinement process in order to prove as early as possible an intended 
(probabilistic) property. 

The efficiency provided by Mtbdds to store and logically manipulate the 
state space made them also the choice of recent quantitative model checkers PI 
10 ]. However, if it comes to model analysis via numerical recipes like simplex or 
(iterative) solutions of equations systems, experience has shown that Mtbdds 
do not outperform classical data structures (such as sparse matrices) [tf 1 1 819 j . 
The main reason appears to be that any of these algorithms tend to require the 
storage of a distinct real number per actual state m- In our case, the use of 
Mtbdds is focus on the manipulation of probabilistic transition relations and 
its use in the abstraction techniques. After abstraction, the size of the problem 
submitted to numerical analysis becomes a significantly smaller issue. 
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Other quantitative model checkers have also been developed. The tool Prob- 
Verus m allows to check the validity of a PCTL formula m on a (discrete 
time) Markov chain. Therefore, models do not contain non-determinism. Instead, 
Prism P is a quantitative model checker for PCTL formulas on (discrete time) 
Markov decision processes, i.e., non-determinism is inherent to the model. Like 
Prism, we also do model checking on Markov decision processes, but we restrict 
to a particular kind of PCTL formula. For completeness reason, we also mention 
the quantitative model checker E I— M pzi, which model checks probabilistic 
timed properties on continuous-time Markov chains. 

Organization of the paper. SectionEland SectionElintroduce the theoretical foun- 
dations of the implemented tool. The algorithms, data structure, and method- 
ological techniques are explained in Sections El and 0 An example is reported in 
Section 0 Finally we present our conclusions and discuss further work. Proofs 
and further details are reported in 0. 

2 Probabilistic Transition Systems 

Probabilistic transition systems (PTS for short) generalise the well-known tran- 
sition systems with probabilistic information. In a PTS, a transition does not 
lead to a single state but to a probability space whose sample space is a set of 
states. The model we define is widely used (see, e.g. j2EEE31-) and is also known 
as Markov decision processes m We consider in addition a function that labels 
each state with a property assumed to be valid in this state. 

Let Distr(f2) be the set of all discrete probability distributions over the sample 
space fl. Let PF be a set of propositional formulas closed under A and -i. 

Definition 1. A probabilistic transition system (PTS for short) is a structure 
T = {S, — >,f) where S is a set of states, — > C S x Distr(S') is the transition 
relation, and f : S ^ Pf is a proposition assignment. We write s — > tt if 
(s, 7t) G — y, and s — > if there is a w such that s — > tt; otherwise, we write 
s-/-^ and call s a sink state. A PTS is said to be a fully probabilistic transition 
system (FPTS for short) if whenever s — > tt and s — > p then tt = p. It will be 
convenient to distinguish an initial state sq G S. In this case we call the structure 
(T, So), a rooted (fully) probabilistic transition system. A proposition g G PP is 
satisfied in state s, notation s \= g, whenever /(s) => g holds is a tautology. 

Example 1. Consider a system that either increments a 
counter with probability 0.5 or it deadlocks with proba- 
bility 0.5 while the counter is smaller than 20. Formally, 
it can be modelled by a PTS Counter = {S, — >■, /) where 
S' = {a, b} X {0 . . 20}, /(s, z) = (s A a; = i), and 

(a,20)^{(a,20)^l| 

(a,f) — {(a,f -I- 1) !->• 0.5, (b,j) H> 0.5} if i < 20 
A symbolic representation of this PTS is depicted in Fig. 0 □ 



0 



x> 20 



X < 20 0.5 ^ 



0.5 

X++ 



Fig. 1. 
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Let T = (S', — >,f)- A simple path starting from sq G <5' in T is a finite 
sequence of S-states, a = S 0 S 1 S 2 • ■ ■ Sn, where for each 0 < i < n there exists 
7 Tj G Distr(S) such that Si — > iTi and TTi{si+i) > 0. Let a(i) denote the state in 
the j-th position. Let |ct| be the length of a. Let first{a) = (t( 1) and last{a) = 
cr(| CT I). We let s-paths{T) denote the sets of simple paths in T starting from any 
s G S. A state t is reachable from other state s in T if there is cr G s-paths{T) 
with s = first{a) and t = last{a). Let reach{J,s) denote the set of all states 
reachable from s in T. 

For any rooted FPTS (F,s), the probability measure Pp_s on the cr-algebra 
induced by (F, s) is the unique probability measure defined such that Pf,s(o’) = 
if (s = So) then 7 To(si) • 7 Ti(s 2 ) • . . . • TTn-i{sn) else 0. In particular, Pp s(c) is 
the probability of u in F starting from s 

Any given PTS T defines a set of probabilistic executions, each one obtained 
by iteratively scheduling one of the possible post-state distributions from each 
pre-state, starting from a given state sq G S. Notice that the same state s of 
T may occur more than once during a probabilistic execution and each time 
a different distribution from s may be scheduled. In order to distinguish such 
occurrences every state s of a probabilistic execution is extended with the past 
history of s, that is, with the unique path leading from the start state to s. 

Definition 2. A probabilistic path 0 / T is a FPTS F = 

{s-paths{T), — >f,folast) where q — P implies last{q) — tt with 

p{qs) = 7t(s) for all s € S. If in addition, for all q G s-paths{T) such that 
1 9 1 < i, last{q) — >-T implies that q — S^p, then the rooted FPTS (F,sq) is said 
to be a probabilistic execution fragment of length z of T starting from sq G S. 
If i = 00 then (F,sp) is said to be a probabilistic execution of T starting from 
sq G S . 

Denote by pathsfT) the set of all probabilistic paths of T, by execs(T,so,i) 
the set of all probabilistic execution fragments of length i starting from sq, and 
by ea;ecs(T, sp) the set of all probabilistic executions of T starting from sp. 

Given a simple path a G s-paths(T) define crt g s-paths(F) (F being a prob- 
abilistic path of T) such that | crt | = |cr| and for all 0 < i < |cr|, a'^(i) = 
cr(l) . . . cr(i). We extend t to sets of simple paths in the usual way. Let f G PF 
and define Sf = {a G s-paths{T) \ last{a) |= f and VO < z < |(t|. ct(z) ^ -if}, i.e., 
Sf is the set of all minimal paths in T that end in final condition f . The minimum 
and maximum probabilities of reaching a final condition f G PF from an initial 
condition i G PF in a rooted PTS (T, sp) are defined respectively by 

P™s„(i,f) = inf {Pp^q)!!/) | s G reach{T,so), s |= i, and (F,q) G execs(T,s)} 

Pt“so(fO = sup {Pp,q(Af^) I s G reach{T,so), s |= i, and (F,g) G execs{T,s)} 

Example 2. Consider the Counter of Example d Take the initial condition i = 
(a A a: = 0) and the final condition f = (b A a: > 15). The reader is invited to 
check that Up = {(a, j){a, j + 1) ... {a, i — l)(a, i){b,i) \ 15 < z < 20 A 0 < j < i} 
and that PV"'(,,p)(i, 0 = PxTa.o)(Ff) = ° 
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3 Probabilistic Simulation 



Probabilistic simulation will be central to state the correctness of the 

technique proposed in this paper. 



Definition 3. Let C Q S x S be a relation on states defining a discrimination 
criterion. R is a C - (probabilistic) simulation if whenever sRt, 

1. (s, t) G C, and 

2. if s — > 7T, there exist p such that t — > p and tt Qh p. 

where tt p if there is S € Distr(S' x S) such that for all s,t € S, (i) tt{s) = 
5{s,S), (a) p{t) = 5{S,t), and (Hi) 6{s,f) > 0 => sRt. s is C-simulated by t, 
notation s <c t, if there is a C -simulation R with sRt. 



Notice that whenever the discriminating criteria C is a preorder, so is <c- 
Our interest is to check when a PTS reaches a goal f starting from any 
state satisfying some initial condition i (i,f G PF). Consider the discriminating 
condition (s,t) G C;y defined by (s |= f t \= f) and (s ^ i 4=^ ^ h 0 We 
write only C whenever i and f are clear from the context. Simulations <c, 
and <cnC-i Eire the relations needed to prove correctness of the technique. 

We are interested in whether the probability of reaching a particular final 
condition f from any (reachable) state satisfying a given initial condition i is 
smaller or greater than a given value p. The next theorem states that if a PTS 
Ti satisfies this property, and another T 2 (C fl C“^)-simulates Ti, then T 2 also 
satisfies the property. 

Theorem 1. Let (Ti,sJ) and (T 2 ,Sg) be two rooted RTSs such that none of 
them contains a sink state. Then 



1- (Ti,sJ) <c (T2,s§) implies 

2. (Ti,sJ) <c-i (T2,s§) implies P“f^i(i,f) > P“f^ 2 (i,f). 

5. (Ti,sJ) <cnc-i (T2 ,s§) implies Pi^7,i(i,f) <p\l\.{(f) and PVt«j(i,f) > 



The requirement that every state has a transition is not really harmful as 
each sink state can always be completed with a self-looping transition without 
affecting the properties of the original PTS. 

The proposed technique is based on successive refinements of a coarse ab- 
straction of the original PTS T. Each refinement is an abstraction of the next 
(finer) refinement in which T is the finest one. In the following, we state that 
the refinement operation preserves simulation. A consequence of Theorem Q is 
that if a given abstraction satisfy the desired reachability property, so does T. 



Definition 4. Let A = {Ai)j be a partition of S, i.e., for all i,j G I, AiC\Aj 
0 4=^ i = j, and IJ^ Ai = S. Let T = (S', — >, /). The quotient PTS according 
to A is defined by T jA = {A, — /a)? where 
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1. A — if 3s G A. s — )> 7T and VA' G A. {t:IA){A') = X^s'gA' 

2 - /^(A) = A.ga/W- 

For a rooted PTS (T, sq), its quotient is given by (T, stf)/ A = (T /A, A) provided 
sqG Ag A. 



We say that F jA is a C-abstraction of T if for all A G A, and s,t G A, 
(s ^ i 4=^ t 1= i). We say that T jA is a (C fl C~^) -abstraction of T if for all 
A G A, and s,t G A, (s ^ f 4=^ t ^ f), and (s ^ i 4=^ t \= i). 

Theorem 2. Let (T, sq) be a rooted PTS. Let B be a refinement of A (i.e., B 
is also a partition of S such that \/B G B. 3 A G A. B G_ A). 

1. if T/ A is a C-abstraction of T then (a) (T, sq)/B <c O', sq) / A, and (b) T /B 
is also a C-abstraction ofT. 

2. ifT/Aisa{Cr\C~^)-abstractionofT then (a) (T,sq)/B <cnc~^ O,^o)/A, 
and (b) T /B is also a (C fl C~^)-abstraction ofT. 

Example 3. Let 



{{(a,0)},{(a,A I l<i< 15},{(a,A I 15 < i < 20 }, {(a, 20 )}, 
{(b,i) I 0 < z < 15},{(b,*) I 15 < z < 20},{(b,20)}} 



Then Counter/^ = (A, — >A,fA), with — de- 
fined by (see also Fig.|^ [(a,o)] (g 



[(a, 0)] {[(a, 1)] ^ 0.5, [(b, 0)] ^ 0.5} 

[(a, 1)] {[(a, 1)] ^ 0.5, [(b, 0)] ^ 0.5} 

[(a, 1)] {[(a, 15)] ^ 0.5, [(b, 1)] ^ 0.5} 

[(a, 15)] {[(a, 15)] ^ 0.5, [(b, 15)] ^ 0.5} 

[(a, 15)] {[(a, 20)] ^ 0.5, [(b, 15)] ^ 0.5} 

[(a, 20)] ^^{[(a,20)]^l} 



[(a.l)] 
[(a, 15)] 




[(b,0)j 

[(bT5)j 



[(a, 20)] 



id 



O [(t’’20)] 



where [s] denotes the class of s, i.e., the set in A 
such that s G [s]. Notice that Counter/^ is in- 
deed a (C n C“^)-abstraction of Counter with i and f as before. In addition, 
^L[(a.O)] (i,f) = 0 and P} 7 (,, 0 )](' , f) = j, which is a sound approximation of the 
actual solution (see Example 0 . □ 



4 Model-Checking and Partitioning 

To perform model checking, the require the PTS under study to be finite. In 
order to describe the encoding of PTS we need some particular notation. If 
s — > 7T, we call the pair (s,7t) a nail. Nails have the same functionality as 
probabilistic states in alternating models; they are depicted with black boxes in 
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Fig.0 Let T = {S, — >, f) be a PTS. From now on we assume the initial and 
final conditions i and f are atomic propositions and for all s € S, f{s) is either 
true, i, f or i A f . T will also be described in an equivalent manner by a tuple 
{S, N, Org, T, /) where N is the set of nails, Org : N ^ S associates to each nail 
(s,7t) its origin state s, and r : TV — >■ Distr(S') associates its distribution tt. Let 
Af{s) = {n £ N \ Org(n) = s} be the set of outgoing nails of state s. Notice 
that nails having different origin states can share the same distribution. 

Computing P“^(i) and P®“P(i). For the rest of this section we wil use the 
shorthand P“^(s) and P®“p(s) for Pjg(s,f) and Pj“^(s,f), respectively. 

According to 0, the sets of states for which P‘"^(s) = 0 or P®'^p(s) = 0 can be 
computed by resorting to simple fixpoint computations on graphs, whereas the 
infimum and supremum probabilities of the other states satisfies the following 



equations: 






P“f(s) 


= min 

n£N {s) 


(1) 


P™P(s) 


= max 

n£j\f{s) 


(2) 



To solve such a system, two methods have been explored: one can either trans- 
form such a system into a linear programming problem, and use classical tech- 
niques of linear programming, or consider the system as a fixpoint equation, and 
compute its least fixpoint by iterative methods. The solving method is however 
not the aim of this paper. We choosed linear programming with exact arithmetic, 
in order to avoid numerical problems and to get exact results. 

Partitioning and complexity of the analysis. Basically, the two sources of 
complexity in these systems of equations are first the number of states of the 
PTS, which is of the same order as the number of variables, and then the number 
of nails, which gives the number of linear expressions in min or max expressions. 

Partitioning the state space allows to address the first source of complexity. 
The question is how to proceed with the nails. Consider the PTS depicted in 
Fig.ETa). The first effect of the abstraction is that several edges outgoing from 
a nail will lead to the same class; we have to merge these edges and add their 
probabilities, as shown on Fig. Hb), where So,Si and S 2 ,s^ are merged into 
equivalence classes fcg and k\. A second effect is that this operation makes some 
nails become equivalent, as (so,ao) and (sojOi) on Fig. 0(b). This effect is our 
main point: we expect that partitioning the state space will equate many nails 
and therefore address the second source of complexity. Such a situation is very 
likely to happen in systems that are specified in a symbolic way using data 
variables (see example 0) . 

5 Algorithms and Data Structures 

Representation of states, transition relation, and partitions. As stated 
in the introduction, we use Bdds to represent sets of states, and Adds to rep- 
resent the transition relation. 



46 



P.R. D’Argenio et al. 




As S is finite, we can encode each state s G S' by a Boolean vector s of 
length n = |"log 2 |S|] . We then use Bdds to represent (sets of) states. Similarly, 
we use Adds to represent the function r, which belongs to the space — >■ (S — ?> 

[0, 1]), isomorphic to the space A x S — >■ [0, 1]. In order to encode nails, we use 
an auxiliary set A that solves in each state the nondeterministic choice on its 
outgoing nails. Let p = log(maxsgs |A/’(s)|) and B = {0,1}. We consider that 
S C B”, A C BP, N C S X A, and r : S x A x S — >■ [0, 1] . r is then represented by 
an Add using unprimed variables, auxiliary and primed variables, noted ~a^, 

. The Boolean vectors s and s' represent states s and s', using respectively 
unprimed and primed variables. Each nail n G A/"(s) outgoing from a state s is 
thus encoded by a pair (s, a) G S x A. Fig. Efa) shows the Add representing the 
system of Fig. Ha). 

A partition of a finite set S is defined by a set K of classes, and a function 
def : AT — >• 2'® such that: UfcGif ^ and Mk ^ k' ■. def{k) fl def{k') = 0 

(and Vfc : def{k) 0). As usual, sets are represented by Bdds. In order to use 
classes k in Bdds, we use Boolean vectors fc, k' G B”, represented with variables 
^ and k^. 

Simplification of a PTS and Boolean analysis. Before performing any 
abstraction, we first try to simplify the PTS, using conditions i and f. Obviously, 
states that are not reachable from states satisfying i cannot influence the value of 
P™^(i) and P®'^P(i), so we can discard them from S and simplify r. In a different 
spirit, the states s satisfying P‘"^(s) = P®“p(s) = 0 or P“^(s) = P®“p(s) = 1 can 
be respectively gathered in a safe partition or included in the final partition. 
These partitions are then transformed in a sink state by appropriately changing 
r, since their outgoing transitions are irrelevant for the computation. Afterwards, 
a new reachability analysis allows to further simplify the state space and the 
transition function. The computation of such states can be done by fixpoint 
computations with Bdds, by considering a suitable Boolean abstraction of a 
PTS. 

Notice that we do not necessarily reduce the number of nodes of Bdds and 
Adds by the above simplifications. However, futile computations are avoided by 
restricting partitioning only to the relevant states. Boolean fixpoint computa- 
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Si 




(f) 



The states sq,si,S 2 ,S 3 are encoded by Boolean vectors on 
variables (so, si), so being the least significant bit; ao and ai 
are encoded by variable a, and class fco and fci by variable k. 
The left branch of a node corresponds to a false value for its 
variable. For readability, leaves are sometimes duplicated. 
The nodes 7r“ and vrf represent respectively the distribu- 
tions {fco e-)- |;fci 1 -^ |} and {ko i-)- 0; fei 1}. We have 
g{ko, nf) = -.So A -.si and g{ko, ttj ) = isi. 



Fig. 4. Representation and abstraction of the PTS of Fig. |^with Adds 



tions are also used on PTS abstracted by partitioning in order to compute the 
set of classes k for which P™^(fc) = 0 or = 0, which is required to solve 

equations CO) and (0 in Section H 



Abstraction of a PTS by partitioning. The problem is the following: given 
a system {S,N C S x A, Org, t) and a partition {K, def) of S, compute an 
abstract system {S°‘ , N°‘ , Or^ , t°‘) with 

= K , N°‘ = N , : N Distr(A) 

(s, a) ^ \k.{Y.s'ddef{k) «')) 

We do not specify the function Org“ since it is not necessary to compute it. 

To compute r“ with Adds, we first transform the summation indexed by 
s' G def{k) into an unconstrained summation. For k' G K, define the Add 

T^k'{s,a,s') = ite(s' G def{k'),T{s,a,s'),0) 

where ite is the if-then-else operator on Adds (see Figs. 0b) and (d)). Then, for 
every (s, a, k'), r°‘{s,a, k') = ct, which we note a). This 

unconstrained summation on all valuations taken by primed variables correspond 
exactly to the existential quantifieation of primed variables in the Add T-^k ' , as 
defined for instance in the library Cudd 03- This operation benefits from the 
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usual caching techniques of Bdds, and can be implemented in a time quadratic 
in the number of nodes of the input graph. So we have 

and T°‘{s,a,k') = Efc'eif 0) 

where ^ disjoint summation on Adds implemented by a cascade of ite 

operators (see Figs.S^c), (e), and (f)). Computing r“ in that way requires \K\ 
intersection operations (to obtain r^k), \K\ existential quantifications on Adds, 
and \K\ applications of the ite operator. 

From ADDs to equations on abstract system. Let {S°^ , N°^ , Or^ , t°‘) be 
an abstract system defined by a partition with initial class k\ and final class fcf. 
We want to compute P“^(fc|) and P®'^P(fc|). Therefore we need to generate the 
systems of inequalities (P) and m from the Add r“. That is, to each class k, 
we have to associate its set of outgoing nails {(s,a) | s S def{k)}, extracting 
the corresponding distributions and detecting efficiently identical distributions 
to avoid redundancy in equations. 

We select the nails outgoing from a class k by computing 

a, k') = ite(s G def{k),T°‘{s, a, fc'), 0) 

The important point now for the extraction of the distributions is that we require 
the variables Ic to be ordered below the variables ~s^ and in Adds. This 
allows to extract the distributions by performing a depth-first search of the graph 
rooted at r^, stopping as soon as a node indexed by a variable belonging to 
is encountered. Such a node corresponds to a distribution (Fig. Pf)). Because 
of this variable ordering and the sharing of nodes the Add the set of its 
different distributions can be obtained for free by a simple graph algorithm. 

The third step, generating a linear expression from an Add representing a 
distribution, is done by enumerating the valuations on variables that leads to 
a non-zero leaf, c.f. Fig. Pf) and the explanations. We resort then to section 0 
to solve the system of equations. 

Automatic partition refinement. The choice of a suitable abstraction is a 
difficult problem, because only the results of the analysis can decide whether the 
abstraction offers enough precision to check the intended property. This is why 
we have chosen an incremental partitioning method. 

The verification starts with a rough partition of the system. If the analysis of 
this abstract PTS allows to conclude that the property is satisfied by the concrete 
PTS, the verification process is finished. Otherwise, a partition refinement step 
is performed in order to obtain more precise information. This process is iterated 
up to success or until all classes of the partition are stable. If this last situation 
occurs, we can conclude that the property is false and extract a counter-example 
path. 

The initial partition contains three distinguished classes: the safe, initial, and 
final classes, denoted fcs, k\, and kf, with def{ks) = {s | P’*“p(s) = 0}, def{ki) = i. 
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and def{kf) = f. The safe and final classes are never split. As our tool allows 
to specify processes that combines an explicit control structure and operations 
on data variables, we use this control structure to partition the remaining state 
space. 

Our refinement method tries to stabilize classes, by separating concrete states 
in a class that have different future, as do all partition refinement methods 
based on a bisimulation criteria mi, and most of those dealing with infi- 
nite state systems mmr\ . We implement this idea by considering the set of 
states g{k,TT°‘) C def{k) in class k that can lead to the abstract distribution 7t“. 
g{k,7T°‘) can be seen as the guard of the distribution 7 t“ in class k. For instance, 
in Fig. 0 b), if 7 rf denotes the distribution attached to the nail (so,flo)> then 
( 7 (^ 0 , 7 t“) = {so}, and if 7 T 2 denotes the distribution associated to nails (so,ai) 
and (si, flo), g(fco, ^2 ) = {^o, Si}- If such a guard is neither empty, nor equal to 
the definition of the class k, then the class k can be safely split into two classes 
k' and k” according to this discriminating guard, with def{k') = g{k,TT°‘) and 
def{k”) = def{k) \g(fc,7r“), because states in class k' are certainly not bisimilar 
to states in class k". A guard g{k,n°') is obtained by computing the union of 
paths in the Add that leads from the root node to the node representing the 
distribution 7r“ (Fig. 0f)). Such an operation can again be implemented with a 
complexity linear in the number of nodes of the Add 

Our global strategy for refinement tries, between each analysis step, to split 
once every class for which there exist a guard. After a partition refinement, a 
new abstract transition function r“ has to be computed. When a class k has not 
been split, the Adds r^k and are reused; otherwise, we need to recompute 
them, as well as the Adds t“ and t^. So the refinement process require 0{\K\) 
Bdds operations. 

Conclusion. The algorithms presented in this section allows to partition and 
to refine an abstract PTS with 0{\K\) Bdds operations; the complexity of these 
operations is in turn linear or quadratic in the number of nodes of the input 
diagrams. 

6 Example 

The Bounded Retransmission Protocol (BRP) [1 1 511 217 j has become a nice bench- 
mark example as it is simple to understand, yet its overall behaviour is not trivial. 
The BRP is based on the alternating bit protocol but allows for a bounded num- 
ber of retransmissions of a chunk, i.e., part of a file, only. So, eventual delivery 
is not guaranteed and the protocol may abort the file transfer. By using our 
technique, we are able to quantify the probability of such abortion. 

The protocol consists of a Sender and a Receiver exchanging data via two 
unreliable (lossy) channels, K and L. The Sender reads a file to be transmitted 
(which is assumed to be divided in N chunks) and sets the retry counter to 0. 
Then it sends the elements of the file one by one over K to the Receiver. A frame 
sent through channel K consists of three bits and a chunk. The first bit indicates 
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Sender: <^MAX) 

f, := (i = 1), Is := (i = N) 
bs := ab, nrtr-\ — \- 




Receiver: 




Channel K: Channel L: 




Fig. 5. PTS model of the bounded retransmission protocol 
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whether the chunk is the first element of the file, the second one indicates if it is 
the last one, and the third bit, the so-called alternating bit, is used to guarantee 
that data is not duplicated. After sending the frame, the Sender waits for an 
acknowledgement or for a timeout. In case of acknowledgement, if it corresponds 
to the last chunk, the sending client is informed of correct transmission (signal 
OK); otherwise the next element of the file is sent. If a timeout occurs, the 
frame is resent (after the counter for the number of retries is incremented), or 
the transmission of the file is broken off. The latter occurs if the retry counter 
exceeds its maximum value MAX. In this case the sender client is informed 
whether the Sender did not complete the transmission (NOK), or whether it 
sent the last chunk but it was never acknowledge (DK) in which case the success 
of the transmission is unknown. Afterwards, and before sending a new file, the 
Sender waits enough time to ensure that the Receiver has properly reacted to the 
communication break. 



The Receiver waits for a frame to arrive. This frame is delivered at the re- 
ceiving client informing whether it is the first (FST), an intermediate (INC), or 
the last one (OK). Afterwards, an acknowledgement is sent over L to the Sender. 
Then the Receiver simply waits for more frames to arrive. The receiver remem- 
bers whether the previous frame was the last element of the file and the expected 
value of the alternating bit. Each frame is acknowledged, but it is handed over 
to the receiving client only if the alternating bit indicates that it is new. Note 
that (only) if the previous frame was last of the file, then a fresh frame will 
be the first of the subsequent file and a repeated frame will still be the last of 
the old file. If a long enough time had passed since the last frame was received, 
the Receiver assumes that the normal communication flow broke down. If this 
happen, the receiving client is informed, provided the last element of the file has 
not yet been delivered. Since our model does not consider time, we assume that 
premature timeouts are not possible and that the Sender and Receiver always 
re-synchronise properly after normal communication is broken. 



The description of the components of the protocol in terms of PTS is given 
in Fig. 13 It abstracts from the data that is being transmitted. The components 
synchronise through common alphabet (a la CSP [El). Notice that the only 
probabilistic features are those occurring in the medium. In this model we assume 
that a frame is lost with probability 0.02, and acknowledgement is lost with 
probability 0.01. 



NewFile 
T false 

0 NewFile 
T true 



o 



A checking automaton (Fig. EJ ensures that the trans- 
mitted file is invariant for the property under study, i.e, 
the property is only interesting for exactly one file trans- 
mission. Notice that the checking automata selects an arbi- 
trary file to test. We study several properties. The consid- 
ered initial condition is test@Check A next_frame@Sender A 
(iQSender = 1). The different final conditions are listed in Fig. 6. 

Table E Notice the flag recv at the Receiver side; it is used to register that 
the last sent file has actually started to be received. Properties A and B define 
the minimal correctness requirement of the protocol. They should not be valid. 



any 
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Table 1. Reachability Conditions 





Final condition 


Meaning of the property 


A 


(srepOSender = NOK) A 
(rrepO Receiver = OK) A 
(rec?;@ Receiver) 


The Sender reports a certain unsuccessful 
transmission but the Receiver got the complete file. 
(The probability should be 0!) 


B 


(srepOSender = OK) A 
-i(rrep@Receiver = OK) A 
(recr;@ Receiver) 


The Sender reports a certain successful transmission 
but the Receiver did not get the complete file. (The 
probability should be 0!) 


1 


errorOSender 


The Sender does not report a successful transmission 


2 


errorOSender A 
(srepOSender = DK) 


The Sender reports an uncertainty on the success of 
the transmission 


3 


errorOSender A 
(srepOSender = NOK) A 
(iOSender > 8) 


The Sender reports a certain unsuccessful 
transmission after transmitting half a file 


4 


-i(srepOSender = T) A 
-i(rec?;©Receiver) 


The Receiver does not receive any chunk of a hie, i.e., 
the hrst message never arrives. “-i(srepOSender = T)” 
ensures that the Sender did try to send a chunk 



Properties 1 to 3 are concerned with transmissions that the Sender does not 
consider successful while property 4 considers an attempt for transmission with 
no reaction at the Receiver side. 

The exercise we perform is to try to find the minimum number of retrans- 
missions {MAX) that satisfies our probabilistic requirements for these properties 
when the transmitted file has length N = 16. TableQ reports these results. Some 
remarks are in order. Each row in Table |2I reports a different instance according 
to the maximum number of retransmission MAX which is specified in the first 
column. The second column reports the number of reachable states in the re- 
spective instance (preach.), and the third one the number of relevant states, i.e., 
reachable states that may lead to a state satisfying the final condition (^^^relev). 

For each property we tried two different values of desired probability. Thus, 
for instance, property 1 is require to hold with probability less than 0.05 in the 
first experiment and than 0.01 in the second one. The least three columns report 
the last possible refinement together with its respective convergence value. For 
each experiment we report the number of refinements (#refin.) necessary to con- 
clude the required property and for this last refinement, the number of abstract 
states (^abst.) and the upper bound for the actual minimum and maximum 
probability (P‘"^ and P®“p, respectively). We also report whether the property 
holds (fy) or not (x) on the verdict columns (Verd.) 

Notice that, in this example, the proposed method do the actual verdict 
on an abstract state space which is, on average, around 20 times smaller than 
the concrete reachable state space. In particular it has performed quite well for 
Properties 2 and 3 in the larger systems {MAX > 3). We have experience two 
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Table 2. Results in a BRP with file length = 16 
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c’ 

o 

i. 




pinf _ pSup 


Property 1; 




prob. < 10 ^ 




prob. < 10 ® 




Convergence | 


0 


1174 


269 




1 


12 


0.0298 


1 


X 




1 


12 


0.0298 


1 


X 




88 


99 


0.383717 


1 


2068 


697 




6 


35 


1.5679e-03 


1 


X 




1 


17 


6.01899e-04 


1 


X 




92 


247 


0.0141144 


2 


2962 


1125 




92 


404 


4.22546e-04 


4.24145e-04 


V 




3 


27 


1.20404e-05 


1 


X 




94 


411 


4.23333e-04 


3 


3856 


1553 




94 


564 


1.25943e-05 


1.2642e-05 


V 




76 


459 


1.02284e-05 


1 


X 




96 


575 


1.26178e-05 


4 


4750 


1981 




95 


724 


3.75311e-07 


3.76733e-07 


V 




95 


724 


3.75311e-07 


3.76733e-07 


V 




97 


739 


3.76012e-07 


5 


5644 


2409 




95 


884 


1.11843e-08 


1.12267e-08 


V 




95 


884 


1.11843e-08 


1.12267e-08 


x/ 




97 


903 


1.12051e-08 


Property 2; 




prob. < 10 ^ 




prob. < 10 ® 




Convergence | 


0 


1174 


1134 




85 


99 


0.0189293 


0.0189293 


X 




85 


99 


0.0189293 


0.0189293 


X 




85 


99 


0.0189293 


1 


2068 


2028 




87 


244 


8.76261e-04 


8.76307e-04 


X 




87 


244 


8.76261e-04 


8.76307e-04 


X 




89 


247 


8.76284e-04 


2 


2962 


2922 




8 


54 


0 


2.64633e-05 


x/ 




89 


404 


2.64531e-05 


2.64531e-05 


X 




91 


411 


2.64531e-05 


3 


3856 


3816 




9 


72 


0 


8.88033e-06 


V 




10 


76 


0 


7.88615e-07 


V 




93 


575 


7.88606e-07 


4 


4750 


4710 




9 


81 


0 


2.64636e-05 


V 




11 


93 


0 


2.64636e-07 


V 




96 


739 


2.35007e-08 


5 


5644 


5604 




9 


82 


0 


2.64636e-05 


V 




11 


97 


0 


7.88615e-07 


V 




99 


903 


7.00322e-10 


Property 3; 




prob. < 10 ^ 




prob. < 10 ^ 




Convergence | 


0 


1174 


950 




43 


93 


0.149825 


0.149825 


X 




43 


93 


0.149825 


0.149825 


X 




43 


93 


0.149825 


1 


2068 


1844 




45 


229 


6.15567e-03 


6.15599e-03 


X 




45 


229 


6.15567e-03 


6.15599e-03 


X 




47 


232 


6.15584e-03 


2 


2962 


2738 




42 


360 


0 


1.85196e-04 


x/ 




47 


379 


1.85191e-04 


1.85191e-04 


X 




49 


386 


1.85191e-04 


3 


3856 


3632 




59 


519 


0 


5.52026e-06 


V 




59 


519 


0 


5.52026e-06 


V 




68 


540 


5.52026e-06 


4 


4750 


4526 




43 


371 


0 


3.04092e-04 


V 




46 


379 


0 


1.64505e-07 


V 




102 


693 


1.64505e-07 


5 


5644 


5420 




52 


455 


0 


8.8846e-06 


V 




52 


455 


0 


8.8846e-06 


x/ 




128 


848 


4.90225e-09 


Property 4; 




prob. < 10 ^ 




prob. < 10 ^ 




Convergence I 


0 


1174 


256 




1 


4 


0.02 


0.02 


X 




1 


4 


0.02 


0.02 


X 




1 


4 


0.02 


1 


2068 


465 




1 


6 


4e-04 


4e-04 


x/ 




1 


6 


4e-04 


4e-04 


X 




1 


6 


4e-04 


2 


2962 


674 




1 


6 


0 


4e-04 


V 




3 


8 


8e-06 


8e-06 


V 




3 


8 


8e-06 


3 


3856 


883 




1 


6 


0 


4e-04 


V 




3 


8 


0 


8e-06 


x/ 




5 


10 


1.6e-07 


4 


4750 


1092 




1 


6 


0 


4e-04 


V 




3 


8 


0 


8e-06 


x/ 




7 


12 


3.2e-09 


5 


5644 


1301 




1 


6 


0 


4e-04 


V 




3 


8 


0 


8e-06 


x/ 




9 


14 


6.4e-ll 



Table 3. Performance (with time format “h:mm:ss.d”) 





Property 1 


Property 2 


Property 3 


Property 4 


% 


(< 10”®) 


Converg. 


(< 10“®) 


Converg. 


(< 10“®) 


Converg. 


(< 10“®) 


Converg. 


0 


0.5 


11.6 


13.3 


13.5 


8.7 


8.8 


0.4 


0.4 


1 


0.6 


2:25.6 


3:54.1 


4:11.9 


1:49.4 


2:02.2 


0.5 


0.5 


2 


1.2 


8:27.8 


11:02.8 


11:24.8 


5:16.9 


5:55.2 


0.6 


0.6 


3 


8:50.5 


17:05.8 


6.3 


22:44.9 


11:29.2 


15:30.7 


0.7 


0.7 


4 


27:19.9 


28:52.1 


8.8 


35:50.7 


5:58.0 


40:52.9 


0.7 


0.8 


5 


41:09.3 


45:05.0 


10.2 


52:21.0 


11:03.0 


1:31:14.7 


0.8 


0.9 



different situations: either there is a gradual convergence to the infimum, but 
almost none to the supremum until an abrupt convergence in the last refinements 
(e.g. Property 1), or vice-versa (e.g Props. 2 and 3). The first case case may 
allow for an early rejection of the required property but would require many 
refinements if the property does hold (compare the number of refinements and 
abstract states of the x cases against the ^ cases in Property 1). Instead, the 
second case will give an early report if the property holds (compare now the 
different results in Property 2). 

The exercise of convergence is more costly as no criterion to stop the exe- 
cution is provided and it proceeds until no more refinement is possible or the 
probability has definitely converged. At this maximum point notice that the 
state compression ratio is 10 times on average. 
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7 Concluding Remarks 

In this article we introduced an efficient technique for quantitative model check- 
ing. The method relies on automatic abstraction of the original system. This 
allows to significantly reduce the size of the problem to which numerical analy- 
sis is applied in order to compute the quantitative factor of the property under 
study. Since the numerical analysis is the most costly part of the whole process 
this reduction is of high importance. This reduction is achieved first because 
bisimilar states are never distinguished, and secondly because using incremen- 
tal abstraction refinement and confronting the analysis against a desired (or 
undesired) probability allows prompt answers on very compact spaces. 

The execution time is currently not the best as the tool should be optimised. 
Table 0 reports the tool performance for a set of propertied. The current im- 
plementation performs numerical analysis using linear programming techniques 
under exact rational arithmetics. This method is very fast (compared to the 
painfully slow iterative methods) and it does not suffer of numerical unstabil- 
ity since numbers are represented in its exact form. Two remarks are in order. 
First, numerical analysis is applied in each refinement step, which is inefficient 
since a refinement step may add only few partitions with low chances of sensi- 
bly affecting the result of the previous iteration. Second, the already mentioned 
asymmetric convergence in which only the minimum or the maximum gradually 
converges to the actual value while the other does not until the last refinements. 

It is in our near future plans to develop efficiency improvements. One of these 
improvements concerns the refinement strategy and the suitable alternation of 
refinement and analysis that should be used. Another improvement would be to 
take advantage of the fact that probabilities usually appears only in some part 
of the modelled system: failures do not appear everywhere! 

On a long term agenda, we plan to use this incremental refinement technique 
to check probabilistic timed automata. Model checking of PTCTL properties 
on such model was proven decidable by resorting to their region graphs m 
However, region graphs are known to be impractical. Our technique would allow 
to generate progressively a minimal probabilistic model, in the spirit of 

Acknowledgements. We thank Holger Hermanns and Joost-Pieter Katoen for 
fruitful discussions. 
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Abstract. Recent investigations have shown that the automated veri- 
fication of continuous-time Markov chains (CTMCs) against CSL (Con- 
tinuous Stochastic Logic) can be performed in a rather efficient manner. 
The state holding time distributions in CTMCs are restricted to nega- 
tive exponential distributions. This paper investigates model checking of 
semi-Markov chains (SMCs), a model in which state holding times are 
governed by general distributions. We report on the semantical issues 
of adopting CSL for specifying properties of SMCs and present model 
checking algorithms for this logic. 



1 Introduction 

Model checking is a technique that is more and more used to ascertain prop- 
erties of computer software, hardware circuits, communication protocols, and 
so forth. In this approach, properties are specified via an appropriate tempo- 
ral logic, such as GTL or LTL, while systems are represented as (usually fi- 
nite) state-transition diagrams. More recently, model checking techniques have 
been extended to stochastic processes such as continuous-time Markov chains 
(GTMGs, for short). In particular, efficient verification algorithms have been 
developed for GSL (Continuous Stochastic Logic [,'ll4p5] ). a stochastic variant 
of GTL. GSL supports the specification of sophisticated steady-state and time- 
dependent properties. GTMGs are widely used in practice, mainly because they 
combine a reasonable modelling flexibility with well-established efficient analy- 
sis techniques for transient and steady-state probabilities that form the basis for 
determining performance measures such as throughput, utilisation and latencies. 
The stochastic processes described by GTMGs are characterised by the fact that 
the state holding times, indicating the amount of time the system stays in a 
state, are restricted to negative exponential distributions. As a result of their so- 
called memoryless property, the probability of moving from one state to another 
is independent of the amount of time the system has spent in the current state. 

Although exponential distributions appropriately model a significant number 
of phenomena - related to mass effects - of random nature, in many occasions 
they are inadequate to faithfully model the stochastic behaviour of the system 
under consideration. For example, file sizes of documents transferred via the In- 
ternet, cycle times in hardware circuits, timeouts in communication protocols. 
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human behaviour, hardware failures, and jitter in multi-media communication 
systems cannot be appropriately modelled. In order to model these phenom- 
ena in an adequate manner, general distributions such as heavy-tail HO] (for 
file sizes), deterministic (for cycle times and timeouts), log-normal (for human 
response behaviour EH), Weibull (for hardware failures EO]), and normal distri- 
butions (for jitter [T^) are used. To adopt the model checking approach to these 
distributions, the simplest solution is to approximate general distributions by 
the mean times to absorption of a CTMCs with an absorbing state, represent- 
ing a so-called phase-type distribution. Although the resulting CTMC can be 
analysed using the existing verification algorithms and prototype tools for CSL, 
such approximations (i) easily give rise to a state-space explosion - the number 
of states increases significantly with the accuracy of the approximation and the 
degree of determinism of the desired distribution - are (ii) not easy to handle in 
case of a choice between stochastic delays - a race condition between the entire 
approximated distributions is decisive - and (iii) require the ability to fit the 
desired distribution by an appropriate phase-type distribution - a non-trivial 
problem in general, see e.g., El- 

Therefore as an alternative approach, this paper investigates direct model 
checking of semi-Markov chains (SMCs, for short) |8I1XI . a natural extension 
of CTMCs in which state holding times are determined by general continuous 
distributions. First, the semantics of CSL on SMCs is studied. In particular, 
the formal characterisation of the CSL steady-state operator is adapted as limit 
state probabilities are not guaranteed to exist for finite-state SMCs, in contrast 
to finite-state CTMCs. Instead, the behaviour of SMCs on the long run is char- 
acterised using the average fraction of time the system resides in a state. For 
instance, the formula 5<o.oi (error) is valid in state s iff on the long run for at 
most 1% of the time on average the system is in an error state when starting 
in state s. For finite CTMCs this interpretation is equal to the characterisation 
using the limit state probabilities. Secondly, model checking algorithms are pro- 
posed to verify CSL over finite-state SMCs. Although long-run properties are 
semantically characterised in a slightly different way, they can be checked as for 
CTMCs: a graph analysis to determine the bottom strongly connected compo- 
nents and solving a linear system of equations for each such component suffice. 
(In the literature, strongly connected SMCs are also known as irreducible SMCs.) 
Time-bounded until formulas can be checked, like for CTMCs, by a reduction to 
transient analysis of SMCs. These include probabilistic timed reachability prop- 
erties such as: can the system reach a goal-state within a certain time-bound 
with some minimal (or maximal) probability? Whereas such transient analysis 
for CTMCs can be solved via stable and efficient numerical techniques such as 
uniformisation, for SMCs it requires solving a set of non-trivial Volterra equa- 
tions whose solution algorithms have a worst case time complexity of 
where N is the number of states of the SMC under consideration. 

In the context of logical specification formalisms and automated verification, 
stochastic processes with general distributions have received scant attention in 
the literature so far. Three related works are known to the authors. Van Hung 
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and Chaochen m have defined a probabilistic variant of the duration calculus 
to express properties over SMCs, but did not report on any verification algo- 
rithms. De Alfaro m discusses model checking of long-run average properties 
and expected reachability times on semi-Markov decision processes. These mod- 
els can be considered as SMCs extended with non-determinism. Time-bounded 
formulas are not considered. Kwiatkowska et al. d have recently considered 
the verification of a stochastic variant of timed automata, with clocks that are 
governed by general distributions, against properties in probabilistic timed CTL. 
They show that a finite-state semantics of such timed automata can be obtained 
using the region-based technique where regions are partitioned to cater for 
the stochastic behaviour. Due to the intrinsic complexity of the model checking 
algorithm, it seems practically infeasible. 

Organisation of the paper. Section|2| introduces the basic concepts of SMCs. Sec- 
tional recalls the logic CSL and defines the semantics of CSL over SMCs. Model 
checking algorithms for long-run properties and time-bounded until formulas are 
described in Section 0 Section 0 concludes the paper. 

2 Semi-Markov Chains 

A semi-Markov chain (SMC) can be considered as a Kripke structure in which 
the transitions are labelled by information about the speed at which the chain 
evolves from one state to another. In a SMC, the delay between two successive 
state changes can be generally distributed. This property has to be contrasted 
with continuous-time Markov chains (CTMCs) where these delays need to be 
governed by negative exponential distributions. In this section, we introduce the 
basic concepts of SMCs. A more thorough treatment of SMCs can be found in 

Mm . 



Semi-Markov chains. Let AP be a fixed, finite set of atomic propositions. A 
(labelled) SMC M is a tuple (S', P,Q,L) where S is a finite set of states, P : 
S X S — ?> [0, 1] is the transition probability matrix (satisfying X^s'gS ^') ~ ^ 
for each s), Q : S x S x (IR>o — t [0,1]) is a matrix of continuous probability 
distribution functions (such that P(s, s') = 0 implies Q(s, s',t) = 1), and L : 
S — >■ 2^^ is the labelling function. Function L assigns to each state s £ S the 
set L(s) of atomic propositions a £ AP that are valid in s. 

The intuitive interpretation of a SMC is as follows. There exists a transition 
from state s to s' (which possibly equals s) if and only if P(s, s') > 0. Matrix P 
determines the (discrete) probabilistic behaviour when changing from one state 
to another, i.e., P(s, s') is the probability to move from state s to state s'. Note 
that this is identical to the probabilistic branching of a discrete-time Markov 
chain (DTMC); {S,P,L) is often called the embedded DTMC of SMC M.. Once 
a next state s' of state s has been selected, the state holding time of state s is 
determined according to the probability distribution function Q(s,s',t). Thus, 
Q(s, s',t) denotes the probability to move from state s to s' within at most 
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0.7, E[0.01] 




P 



L(0) = { working } 

L(l) = { sedimented } 

L(2) — { leaking } 

L(3) = {leaking, waiting} 




E[0.01] W[2] E[0.7] 



0.7 0.2 0.1 0 
0.7 0 0.3 0 
0.4 0 0 0.6 

10 0 0 



U[2,5] 1 U[3,7] 

U[l,3] 1 1 

Wf2.5 1 1 



D[4] 



1 

1 



1 



Fig. 1. A SMC describing a boiler. 



t time-units, given that a transition from s to s' will be taken. A state s is 
absorbing if P(s,s) = 1 and Q(s, s) is some arbitrary nontrivial distribution. 
The distribution function H of state s, defined by 



denotes the total holding time distribution in s regardless of which successor is 
selected. 

We assume that the system will stay in a state with at least some non-zero 
probability, or more formally we demand for arbitrary s that there is some t' > 0 
and some e > 0 such that H(s,t') < 1 — e. We further require the mean of each 
state holding time distribution to be finite, i.e., if[H(s)] ^ oo. 

Example 1. As a simple example of a SMC we model a boiler. The system can 
be in four different states, state 0 where the boiler is working properly, state 1, 
where the boiler has too much sediment that needs to be removed, state 2 where 
a pipe is leaking that either needs to be fixed or needs to be replaced, and finally 
state 3 where the system is waiting for a new pipe to arrive for replacement. 
The model is schematically depicted in Fig. [Q together with the matrices P 
and Q (’E’ denotes an exponential distribution, ’U’ a uniform distribution, ’D’ a 
deterministic distribution, and ’W’ a Weibull distribution with appropriate pa- 
rameters) and labelling L. The total holding time distributions can be computed 
from the matrices. For instance 



H(s,t)= ^P(s,s')-Q(s,s',t) 



s'gS 



if t < 2, 



0.23 t- 0.46 
0.31 t- 0.7 
0.08 t- 0.24 
1 



if 2 < t < 3, 
if 3 < f < 5, 
if 5 < t < 7, 
otherwise. 



H(l,t) = 
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To describe how the system evolves from state to state, suppose that the boilers 
starts in state 0. Matrix P immediately determines the probability to move to a 
next state. State 2 is chosen, for instance with probability P(0, 2) = 0.1. In this 
case a sample is immediately drawn from distribution Q(0,2,t) = 1 — *, 

say 5.3. The system thus holds state 0 for 5.3 time units before moving to state 
2. In state 2, again matrix P is used to determine the next successor, say state 
0, whence a random sample is drawn from the distribution Q(2, 0) to determine 
the holding time in state 2 before moving back to state 0. □ 

Paths. Let M = {S, P, Q, L) be a SMC. A sequence sq si S 2 . ■ . , 
with Si € S and ti G IR>o such that P(si,Si+i) > 0 for all i, is called a path 
through M. For path a and j G IN, let a[i] = Si, the (i-l-l)-st state of tr, and 
S{a,i) = ti, the time spent in Si. For t G IR>o and i the smallest index with 
t < J2]=o the state in a occupied at time t. 

Let Path^ denote the set of paths in the SMC M., and Path^(s) the set 
of paths in AI that start in s. The superscript is omitted unless needed for 
distinction purposes. 

Borel space. A probability measure Pr on sets of paths through a SMC is defined 
using the standard cylinder construction as follows. Let sq,... ,Sk G S with 
P(si,Si+i) > 0, (0 < i < k), and Iq,... ,Ik-i non-empty intervals in IR>o. 
Then, C(so, Iqj ■ ■ ■ i Ik-i, Sk) denotes the cylinder set consisting of all paths a G 
Path{so) such that a[i] = Si {i < k), and 5{a,i) £ li {i < k). Let A(Path) 
be the smallest tr-algebra on Path which contains all sets C(s, Iq, . . . , Ik-i, Sk) 
where sq, . . . , Sfc ranges over all state-sequences with s = sq, P(si,Si+i) > 0 
(0 < i < fc), and Iq-, ■ ■ ■ , Ik-i ranges over all sequences of non-empty intervals in 
IR>q. The probability measure Pr on P(Path(s)) is the unique measure defined 
by induction on k by Pr(C(so)) = 1 and for fc > 0: 

Pr(C(so, Iq,-- - , Sk,I\ s')) = Pr(C(so, Iq,... , Sfc))- 

P(sfc, s')-(Q(sfc, s', b) - Q(sfe, s', a)) 

where a = inf /' and b = sup /'. With this definition, a path sq si S 2 . . . 
corresponds to a sequence (sq,0), (si,to)j (S 2 A 0 + ^i); • ■ • of bivariate random 
variables satisfying the properties of Markov renewal sequences This ob- 
servation links our definition of SMCs to the standard definition found in the 
literature. 

On the basis of the probability measure Pr, we can define various measures 
determining the behaviour of a SMC as time passes. For instance, 

7t(s, s',t) = Pr{cr G Path(s) | (T@t = s'} 

defines the probability distribution on S (ranged over by s') at time t if starting 
in state s at time 0. We are particularly interested in two specific measures 
discussed below. 
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1, D[l] 



Fig. 2. A SMG without steady-state. 



First passage time analysis. We are interested in a measure that describes the 
probability 

F{s,s',t) = Pr{(T £ Path(s) | 3t' £ [5(cr, 0),t] . a@t' = s'} 

of reaching state s' for the first time within t time units when starting in state 
s. Note that even if s = s' only paths are considered that leave the state s, since 
t' has to be at least 6{a, 0) which is the time needed to leave s. From we 
have that F{s, s', t) (with s, s' £ S) satisfies the following system of equations: 



F(s,s',t) 



P(s,s') Q(s,s', 




dQ(s, s", a;) 
dx 



F{s" , s', t—x) dx 



Intuitively, the probability to reach state s' from state s for the first time within 
t time units equals the sum of the probability of taking a direct transition from 
s to s' (within t time units) and the probability of moving via some intermediate 
state s" at time x, yet reaching state s' in the remaining time interval t — x. 
It can be proven that this equation system has a unique solution if the state 
holding time for any state in the SMC is positive with nonzero probability (as 
we have assumed) IT51 . 



Long-run average analysis. The long-run average behaviour of a SMC is not 
as homogeneous as it is for CTMCs. In particular the steady-state behaviour 
(usually defined as the limit of 7r(s, s', t) for t — >■ oo) may not exist. 

Example 2. Consider for instance, the SMC depicted in Figure 13 For any t > 0 
the probability 7r(s, s',<) does not equal 7r(s, s',t-|- 1), because the probability 
mass alternates between the two states. Thus, a limit for t ^ oo of 7r(s, s',t) 
does not exist. □ 

However, we can define a related measure based on the average amount of 
time spent in some state, similar to H21. For this purpose, we fix a state s, and let 
(Ts be a path taken randomly from the set Path(s). Then, the quantity ls'(i7s@t) 
is a random variable, indicating whether the state s' is occupied at time t when 
starting in s. Here we use the characteristic function 1 ^/( 5 ") = 1 if s' = s" and 
0 otherwise. 

On the basis of this, we can define a random variable that cumulates the 
time spent in some state s' up to time t (starting in s) by f^. 1 s'(cts@x) dx, and 
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normalise it by the time t in order to obtain a measure of the fraction of time 
spent in state s' up to time t. Since this is still a random variable, we can derive 
its expected value. This value corresponds to the average fraction of time spent 
in state s' in the time frame up to t. For the long-run average fraction of time, 
we consider the limit t — >■ oo, as in pH). 

Definition 1. The average fraction of time T(s,s') spent in state s' on the long 
run when starting in state s is given by: 

t 

J dx 
0 

where Gg ranges randomly over Path{s). 

This measure exists for SMCs whenever the expected values of all the distribu- 
tions Q(s,s') are finite (as we have assumed). Note that for finite CTMCs the 
measure T(s,s') agrees with the usual steady-state limit limt_).oo 7 t(s, s', t). In 
this sense, T conservatively extends the steady-state measure of CTMCs. 



r(s, s') = lim E 



3 CSL on Semi-Markov Chains 



This section recalls the syntax of the continuous stochastic logic CSL, and defines 
its semantics in terms of semi-Markov chains. 



Syntax. CSL is a branching-time temporal logic a la CTL ^ with state- and 
path-formulas based on m 

Definition 2. Let p G [0, 1], ^ C {<,>}, t C lR>o, o,nd a G AP. The syntax 
of CSL state-formulas is defined by the following grammar: 



<P ::= true 



a 



d> AL> 






S<p{<P) 



'P^piT) 



where for t G lR>o path-formulas are defined by 



ip ::= XT> 



‘TU'T 



<L>U-*<L>. 



Other boolean connectives are derived in the usual way, i.e. false = ->true, 

V L >2 = A “'^’ 2)1 and ^1 — >■ <l >2 = ~'^i V L> 2 - 

The intended meaning of the temporal operators U (“until”) and X (“next 
step”) is standard. We recall from jS] the intuitive meaning of U-', V and S: 
The path-formula <l>iU-*<l >2 is satisfied iff there is some x G [0,t] such that L>i 
continuously holds during the interval [0, x[ and L >2 becomes true at time instant 
X. P<p{,<p) asserts that the probability measure of the paths satisfying ip falls 
in the interval lap. The state formula Sap{^) asserts that the long-run average 
fraction of time for a ^-state falls in the interval lap = { 9 G [0, 1] | g < p }. 
Temporal operators like O, □ and their real-time variants 0-* or □-* can be 
derived, e.g. Vapi'O’-' T>) = Vap(trueU-'^ <P) and P>p(D^) = 7^<i_p(0 ->^). 
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Semantics. The state-formulas are interpreted over the states of a SMC. Let 
Ai = {S, P, Q, L) with proposition labels in AP. The definition of the satisfac- 
tion relation |= C S' x CSL is as follows. Let Safc(^) = {sGS|s|=^}. 

s 1= true for all s S S, s |= A ^2 iff s [= <Pi, i G {1, 2}, 

s [= n iff a G L(s), s [= S<p(^) iff Tsat{<P)(.^) ^ 

s 1= iff s ^ s 1= 'P<p{^) iff Prob{s, ip) G I^p. 

Here, Ts'{s) denotes the average fraction of time spent in S' C S with respect 
to state s, i.e. 

Ts,(s) = Y. T(s,s'). 

s'eS' 

Recall that T(s, s') conservatively extends the definition of a steady-state dis- 
tribution for CTMCs. Prob{s, (f) denotes the probability measure of all paths 
a G Path(s) satisfying (p, i.e. 

Prob(s,ip) = Pr{a G Path(s) | cr fy 1 ^}. 

The fact that, for each state s, the set {cr G Path(s) | tr fy (/j} is measurable, 
follows by easy verification. The satisfaction relation (also denoted fy) for the 
path- formulas is defined as usual: 

cr fy iff cr[l] is defined and cr[l] |= <?, 

cr fy iff 3fc > 0. {<j[k\ fy ^2 A VO < i < fc. cr[i] fy <Pi) , 

cr fy (PiU-* <p 2 iff 3a: G [0,t]. (cr@a; fy ^2 A Vj/ G [0,a;[. cr@y |= <?i) . 

4 Model Checking SMCs against CSL 

Model checking SMCs against CSL follows the usual strategy: Given a model 
M — {S, P, Q, L) and a state-formula the set Sat(<P) is recursively computed 
for the sub- formulas of This can proceed via well studied means 1 1 6ldj (on the 

embedded DTMC (S,P,L)) except for the time-bounded until operator , 
and for the long-run operator S. These two operators require specific care. 

Time-bounded until. For computing the probability of satisfying a time-bounded 
until formula, we closely follow the strategy of jO], and reduce the problem to 
a well studied transient measure. More precisely, it will turn out that we can 
compute the time-bounded until probabilities via a first passage time analysis 
in a derived SMC, where certain subsets of states are made absorbing. To this 
end, we let (for SMC Ai and state formula <P) denote the SMC obtained 

from Ai by making all ^-states absorbing. We have: 

Theorem 1. Let Ai = (S,P,Q,L) be a SMC, and T>i and T >2 be CSL state- 
formulas. Then 

Prob^{s,T>iU^*^2) = 

ifs\=^2, 

~ \ Ss'l=g> 2 otherwise. 
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Fig. 3. Satisfaction of F>p(leakingW -‘working) for state 2 and 3 of the boiler example. 



Proof: The proof of the first equality is based on a bijection between the paths 
in M satisfying <PiU-^<l >2 and the paths in V <^ 2 ] satisfying up 

to the state where <p 2 becomes satisfied, and hence over the whole path-prefixes 
contributing to the two probability measures Pr{ cr G Path^ \ a |= <l>iU-*<l >2 } 
and Pr{ tr G | a \= 0-‘^2 }• With respect to the second equality 

we only consider the case s ^ <? 2 - In this case a ^ 0-*<?2 can be shown to hold 
if and only if 3t' G [(5((t, 0),t] . a@t' ^ <^ 2 , since cr[0] ^ <p 2 - The proof follows 
from the definition of F and the fact that ^ 2 -states are absorbing, justifying the 
summation over all <? 2 “States. □ 

Example 3. Returning to the boiler example of Fig, let us check the time- 
bounded until formula P>p(leakingW -‘working). First, we observe that state 1 
does neither satisfy leaking nor working, and hence state 1 does not satisfy the 
path-formula leaking^ -‘working with positive probability. In contrast, according 
to Theorem [0 state 0 satisfies the path formula with probability 1, because 
0 \= working. 

The remaining states 2 and 3 are more interesting. Following Theorem ^ we 
need to investigate a SMC where state 0 and 1 are made absorbing, and com- 
pute the probability of satisfying leaking W -‘working via the values of F(2, 0,t), 
respectively F(3, 0,t) in this SMC. The values of these functions are plotted in 
Fig. El One can see that for pairs {p, t) above the plot the formula is invalid, 
while it is valid for pairs below the plot (and for the plot itself). □ 

While in the above example the values of F can be calculated directly, the 
situation is more involved in general. Recall that F(s, s‘, t) is the unique solution 
of the system of equations 



F(s,s',t) = P(s,s') Q{s,s',t)+ ^ J P{s,s') 

b"^s' i 



dQ(s, s", x) 
dx 



F{s" , s' , t—x) dx. 



This system of equations can be classified as a system of Volterra equations of the 
second type. In principle it is possible to solve them by appropriate numerical 
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methods, such as Volterra-Runge-Kutta methods. A complete guide for these 
methods can be found in [^ ■ A solution of the equations can also be obtained in 
the Laplace domain. This approach works good for small systems and sometimes 
even allows a closed-form solution to be found by hand. For larger systems one 
is faced with two problems. One has to invert a matrix of functions in a complex 
variable, and to reverse the transform to the time domain. 

As described in it™ . the asymptotic space complexity of the latter method 
is 0{N^) and the asymptotic time complexity is 0{N"^) where N = [S'] is the 
number of states. It is therefore not applicable to larger systems. Moreover, the 
numerical Laplace transform inversion can encounter numerical problems under 
some conditions. It is also possible to solve this Volterra system by transforming 
it to a system of partial differential equation, a system of ordinary differential 
equation, initial and boundary conditions, and a system of integral equations. 
Pt contains a comparison of these two approaches together with numerical 
considerations. 

Long-run average. For model checking the operator S^p{<L‘) one needs to accumu- 
late the average fraction of time quantities T(s, s') for each state s' satisfying (L. 
If is a strongly connecte^B SMC, T(s, s') can be obtained via the equilibrium 
probability vector tt of the embedded DTMC {S, P, L), which in turn is given as 
the unique solution of the linear equation system 

7t(s) = ^ P(s^ s) • 7r(s') such that ^ 7t(s) = 1. 
s'gS sGS 



Theorem 2. m Let M = (S', P,Q,L) he a strongly connected SMC, and n be 
as above. Then 



T{s,s') 



7r(s')/r(s') 

J2s"esTis")Tr{s") 



where yi{s") is the expected holding time in state, i.e., p,{s") = if[H(s")]. 

Notice that T(s, s') is independent of the starting state s in this case. If otherwise 
Ml is not strongly connected, we proceed as in |^, and isolate the bottom strongly 
connected subsets of S via a graph algorithm m- Whenever state s' is not a 
member of any bottom strongly connected subset of S, we have T(s, s') = 0. The 
following result allows model checking the S operator in the other cases. We let 
Prob{s, OB) denote the probability of eventually reaching the set B C S from 
state s. This quantity can be computed via the embedded DTMC {S, P, L) [TT)) . 

Theorem 3. Let Ml = (S', P,Q,L) be a SMC, B a bottom strongly connected 
subset of S, and s' € B. Then: 

T{s, s') = Prob(s, O B) ■ T^(s', s') 



where the superscript B refers to the strongly connected SMC Ml^ spanned by B. 



^ A SMC is strongly connected if there is some k such that P*'(s,s') > 0 for each s. 
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Proof: We only consider the case where s B can reach B with positive prob- 
ability. The idea of the proof is to count the average time that the SMC 
spends in class B. Once we have isolated this quantity we are able to compute 
the fraction of time M spends in a particular state of this class. Let Is (s') = 1 
if s' G B and 0 otherwise. We shall calculate the exact value for 



E 



t 

^ J 1b{cts@x) dx 

_ 0 



where as ranges over Path(s). Let t be the time of absorption in B (if as touches 
B otherwise t = oo), t is be a random variable and depends on the path as 
drawn from Path(s). The distribution of t is given by Pr{ t < t' } = F(s, B, t') = 
F(s, s', t') where the latter is the first passage time distribution mentioned 

earlier. 

Since B is bottom strongly connected, the function ls(CTs@a;) will be constant 
1 from t on. So, for t > t we have that 



t 

f t — i 

/ lB(cTs@a:) da; = — ^ 

0 

and otherwise (i.e., t < i) the integral equals 0. So, for fixed t the above integral 
describes a random variable Rt as follows: 



Rt{i) 



^ if t > t, 

0 otherwise. 



The distribution of Rt is 

Pr{ Rt < x} = Pr{ {t — i)/t < x} + Pr{ Rt = 0} 
which can be rewritten, using that F(s,B,x) is the distribution of t, to 
Pr{ Rt < x} = 1 — F{s, B,t — xf) + Pr{ Rt = 0}. 



Now, the expected value E[Rt] is obviously 



f d{l-F{s,B,t-ut)) d(Pr{i?t = 0}) f dF{s,B,t-ut) 

U ; du -b 0 ; = I U t ; du. 



du 



du 



du 



Substituting u = we get 






dF{s,B,y) 

dy 



dy. 



What we are looking for is the limit of this quantity as t — >■ oo given by 
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where E[i] is the expected value of t. Recall that t is distributed according 
to F{s,B,t). Also note that F{s, B,t) = Prob{s,C> B). Since we have 

assumed that state s can reach B with positive probability (and all distributions 
have finite means) E[t\ needs to be finite and hence 



lim E 

t—¥CO 



lB(crs@x) dx 



Prob{s, OR). 



The proof of the theorem follows from this result by two observations. First, 
the time of entering R is a renewal point, i.e., a time instant where the future 
behaviour of the stochastic process does only depend on the currently occupied 
state. Second, the fraction of time spent in a particular state inside R is indepen- 
dent of the starting state - due to strong connectedness - if assuming to start 
inside R. □ 



Example 4- Let us check a long-run average property for the example boiler sys- 
tem, such as 5<p(working). We first observe that the SMC in Fig. [Dis strongly 
connected. Theorem El requires the computation of the expected holding times 
for each state of the SMC, resulting from weighted sums of the involved distribu- 
tions. We get ^(0) = 70.319, ^(1) = 3.95, ^(2) = 3.2, and /r(3) = 0.887. Next, we 
solve the embedded DTMC, and obtain a vector tt = [0.686, 0.1373, 0.109, 0.065]. 
Finally we compute 7sat(working)(s) = 0.981. Since the SMC is strongly connected, 
this value is independent of the state s chosen, and hence 5<p(working) is satis- 
fied (for all states) whenever 0.981 < p. □ 

Apart from the need to derive expected values of general distributed ran- 
dom variables, the numerical algorithms needed for model checking the long-run 
average operator are the same as the ones needed for checking CTMCs |S|. 



5 Concluding Remarks 

In this paper, we investigated adapting CSL model checking to semi-Markov 
chains, an extension of CTMCs in which state holding times are governed by 
general distributions. To achieve a smooth extension of the theory we developed 
an enhanced definition of long-run properties and proved novel results required 
for model checking not strongly connected SMCs. On the practical side, the 
conclusion we draw from our investigation is partially negative: verifying a CSL- 
formula can become numerically very complex when dropping the memoryless 
property. This is caused by the involved procedure needed for checking time- 
bounded formulas such as timed probabilistic reachability properties. We proved 
that long-run properties and (untimed) eventualities can be checked without an 
increase in complexity compared to the CTMC case, though. 

The SMC model considered in this paper incorporates general distributions, 
but is known to be of limited use to model concurrent delays. Compositional 
extensions of SMCs - such as generalised semi-Markov chains or stochastic au- 
tomata EH- are more elegant to apply in this context. It is worth to high- 
light that our practically negative result concerning the model checking of time- 
bounded formulas carries over to these models. Further research is needed to 
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investigate whether abstraction techniques or weaker temporal properties - like 
expected time properties - yield a practical solution for such models. 
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Abstract. Coin lemmas are a tool for decoupling probabilistic and non- 
deterministic arguments in the analysis of concurrent probabilistic sys- 
tems. They have revealed to be fundamental in the analysis of random- 
ized distributed algorithms, where the interplay between probability and 
nondeterminism has proved to be subtle and difficult to handle. 

We reformulate coin lemmas in terms of random variables obtaining a 
new collection of coin lemmas that is independent of the underlying 
computational model and of more general applicability to the study of 
concurrent nondeterministic probabilistic systems. 



1 Introduction 

Coin lemmas are a tool for decoupling probabilistic and nondeterministic argu- 
ments in the analysis of concurrent probabilistic systems [12]. They have revealed 
to be fundamental in the analysis of randomized distributed algorithms [7, 8], 
where the interplay between probability and nondeterminism has proved to be 
subtle and difficult to handle [10]. 

Coin lemmas are formulated in the framework of probabilistic automata [11], 
a probabilistic extension of labeled transition systems (automata) where the 
notion of transition is extended so that a transition from some state s leads 
to a discrete probability distribution over states rather than to a single state. 
Each state enables several transitions, and the choice of which transitions to 
schedule is left unspecified. Nondeterminism is resolved by an entity called a 
scheduler^ and the result of resolving nondeterminism can be described as an 
acyclic Markov process, called a probabilistic execution. Properties that involve 
probability can be studied on probabilistic executions, while typical arguments 
about probabilistic automata involve proving upper and lower bounds on the 
probabilities of events under the action of any scheduler, the events being sets 
of paths in the Markov process that represents a probabilistic execution. 

A coin lemma is a tool to prove upper and lower bounds on probabilities of 
events within any probabilistic execution. Specifically, a coin lemma provides us 
with a rule to associate an event with each probabilistic execution together with 
a lower bound on the probability of the associated events. Typically the events 
associated with a probabilistic execution are derived from events in a known 
stochastic process, and similarly, the lower bounds on probabilities are derived 
from the known stochastic process. The analysis of a concurrent probabilistic 
system is then reduced to the analysis of the events returned by the rules, which, 
being just ordinary sets of paths, do not include any probability. 
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1.1 Coin Lemmas in the Setting of Algorithms 

To understand better the role of coin lemmas we illustrate what happens in 
the area of distributed algorithms, which is the area where coin lemmas were 
proposed first. When studying a distributed algorithm the objective is to state 
that for every scheduler (the entity that decides the temporal ordering of the 
events at different processors) the probability of successful termination is at 
least some number p. That is, translated in terms of probabilistic automata, for 
each probabilistic execution the probability of the event that expresses successful 
termination is at least p. 

In practice it is more convenient to show that a sub-event of successful ter- 
mination has probability at least p, the sub-event expressing the fact that some 
random draws that occur in the algorithm give some specific results. 

Example 1. The randomized algorithm of Lehmann and Rabin for the dining 
philosophers problem [6] works as follows: whenever a philosopher wants to eat 
he/she flips a coin to decide which fork to pick up first, waits for the fork to 
be free and picks it up, and finally checks whether the other fork is free. If 
the other fork is free, then the philosopher picks it up and eats, otherwise the 
philosopher puts down the first fork and starts again. It turns out that if two 
neighbor philosophers draw opposite coins at a certain stage, then the algorithm 
terminates successfully and one philosopher eats. However, the algorithm may 
terminate successfully even if the coins are not opposite. Studying the sub-event 
of successful termination that states that two coins are opposite is enough any- 
way to state that there is a high chance that some philosophers eat eventually. 

In summary, the designer of an algorithm thinks of a stochastic process that 
supposedly occurs in any probabilistic execution, thinks of the fact that whenever 
the stochastic process gives some specific results R the algorithm terminates 
successfully, and concludes that the algorithm terminates successfully with a 
probability that is as high as the probability of R. Unfortunately, in most of the 
cases the stochastic process that the designer has in mind does not take place 
since, for example, the algorithm may terminate before the stochastic process 
completes. 

Since designers rarely consider explicitly the fact that the stochastic process 
they have in mind may not complete, they often come to the wrong conclusion 
that an algorithm is correct whenever it is possible that the stochastic process 
does not complete and at the same time the algorithm does not terminate suc- 
cessfully. 

Example 2. Several randomized algorithms in the literature are based on the 
following argument. The processors communicate with each other and at the 
end of the communication it turns out that some of them have participated in 
a game. If there is a unique winner in the game, then the algorithm terminates. 
The game consists of flipping fair coins until a head comes out and counting how 
many coins are flipped. Thus, the probability of drawing number i is 1/2*. The 
winner is the player who draws the highest number. We know that, given that 
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there are k players, the probability of a unique winner is at least some constant c 
independent of k. Should we conclude that the algorithm terminates successfully 
with probability at least c? 

Although in the literature we can find statements like the one above, the 
correctness of the statement depends on how the number of players is chosen. 
If the players are chosen before starting the game, then the lower bound c does 
hold. Otherwise, we may add players to the game until we obtain two winners, 
which would violate considerably the lower bound c. 

Coin lemmas have the scope of formalizing the arguments that designers 
usually put forward about their algorithms and of avoiding the pitfalls given by 
incomplete stochastic processes. Thus, given a stochastic process, a coin lemma 
provides a mechanical rule to pick up an event in each probabilistic execution, 
the event representing the successful part of the stochastic process, and provides 
a lower bound on the probabilities of the chosen events that is the same as the 
probability of the successful part of the stochastic process. The main problem is 
to decide what event to return whenever the stochastic process does not complete 
in a probabilistic execution. 

Viewing a stochastic process as a sequence of elementary experiments, the 
rule of a coin lemma identifies all the experiments that do take place in an 
execution (a path of the probabilistic execution) and their results. Each path 
in a probabilistic execution appears in the event returned by the rule if it is 
possible to fix the outcomes of the experiments that do not take place so that 
the underlying stochastic process is successful. This idea is called “the essence of 
a coin lemma” in [12], since it is the common denominator of all the coin lemmas 
that have been proposed so far. 

The advantage of this formulation of coin lemmas is that the designer of an 
algorithm is given a set of paths (ordinary executions) that must be shown to 
lead to successful termination if he/she wants to conclude that the algorithm 
is successful with some minimum probability. The executions can be analyzed 
without resorting to probabilities. Furthermore, the rule would say something 
like “if a certain coin is not flipped then the resulting execution must be shown 
to lead to successful termination”. Thus, the designer of the algorithm will be 
forced to prove either that the coin is always flipped or that the algorithm is 
indeed successful whenever the coin is not flipped. 

Example 3. The coin lemma that we should use for the argument of Example 2 
would be associated with the stochastic process where k numbers are drawn. 
Assuming that we have some way to decide which k to use, the scheduler that 
adds players until there are two winners would be discovered immediately since 
in the event returned by the rule we would have several executions where less 
than k numbers are drawn. 

1.2 Our Contribution 

A weak point in the current formulations of coin lemmas [12] is that the rules 
are formulated in a very restrictive language. As a result, many times simple and 
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obvious variations to the rules require the proof of a new coin lemma. In this 
paper we propose a new formulation of coin lemmas based on random variables 
that gives us much more flexibility in the formulation of a rule without forcing us 
to prove new coin lemmas each time. Another advantage of the new formulation 
is that the concept of a random variable is independent of the structure of 
the probability space that represents a probabilistic execution. Thus, the new 
coin lemmas can be used easily on several models. On the contrary, the current 
formulation is heavily based on the structure of the transitions of a probabilistic 
automaton. 

If we express a stochastic process as a sequence of elementary random ex- 
periments, the main objective for the formulation of a rule is to identify the 
places in an execution where each experiment takes place and its corresponding 
outcome. For this reason, we start with the concept of an experiment, which is 
a pair {D, U) where I? is a 0/1-valued random variable that identifies the places 
where the experiment takes place, and [/ is a random variable that identifies the 
successful outcomes. An experiment has taken place in an execution a when- 
ever D{a) = 1. We define the notion of a p-successful experiment, which means 
that the probability of success is at least p whenever the experiment takes place. 
Then, we prove that in each probabilistic execution the probability that either a 
p-successful experiment does not take place or that the experiment takes place 
and gives successful results is at least p. 

The result about p-successful experiments allows us to capture immediately 
the coin lemmas of [12] that deal with single occurrences of actions; however, 
many more general properties can be captured easily like the intricate coin lemma 
necessary for the proof of correctness of the consensus algorithm of Ben-Or [1] 
(see [11] for a proof of correctness based on coin lemmas). The identification 
of the intermediate concept of p-successful experiment and the use of random 
variables turned out to be a considerable improvement over the formalization of 
[12]. 

We elaborate further on the concept of p-successful experiments by showing 
how several experiments can be combined into a single experiment. In this way 
we can capture easily the coin lemmas of [12] that look at the outcome of the 
first experiment that takes place among many experiments. 

Finally, we show how to handle more general stochastic processes composed 
of finite or countable sequences of experiments. We derive two different coin lem- 
mas. In both coin lemmas we consider a collection of experiments {{Di, 
and a measurable function f{Xi,X 2 , ■ ■ .) with values in {0, 1}. Then we iden- 
tify the probability distribution of the A^’s under the assumption that all the 
experiments take place, and we prove a lower bound on the probability of the 
event that considers all those cases where it is possible to fix the outcomes of the 
experiments that do not take place so that / evaluates to 1. Once again, if we 
assume to be successful whenever / evaluates to 1, the principle is that if some 
experiment does not take place and one of its possible outcomes is considered as 
successful, then we should make sure that the system under examination behaves 
correctly. 
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The rest of the paper is structured as follows. Section 2 gives the necessary 
background on probability theory and introduces some notation; Section 3 gives 
a description of probabilistic automata; Section 4 introduces the coin lemmas 
for single experiments and compares them with the formulation of [12]; Section 
5 proves properties of single experiments that allow us to combine several ex- 
periments into a unique new experiment; Section 6 introduces the coin lemmas 
for general stochastic processes; Section 7 gives some concluding remarks. 



2 Preliminaries 

A cr-field over a set A is a set T Q 2^ that includes X and is closed under com- 
plement and countable union. Observe that 2^ is a a-field over X . A measurable 
space is a pair (X,X) where A is a set and A is a a-field over A. The set A 
is also called the sample space. A measurable space (A, A) is called discrete if 
T = 2^. In the paper we work mostly with discrete measurable spaces except 
when we deal with probabilistic executions. 

Given a measurable space (A, A), a finite measure over (A, A) is a function 
/X : A — >■ such that, for each countable collection {Aijig/ of pairwise disjoint 

elements of A, ^(U/Ai) = /x(Ai). For the purpose of this paper we are inter- 

ested in measures pi where pi{X) < 1. A probability measure over a measurable 
space (A, A) is a measure pi over (A, A) such that /i(A) = 1; a sub-probability 
measure over (A, A) is a measure pi over T such that /x(A) < 1. A measure 
over a discrete measurable space is called a discrete measure. We also say that a 
discrete measure over (A, 2^) is a discrete measure over A. Sometimes we refer 
to probability measures as distributions. 

Given a set A, denote by Disc{X) the set of discrete probability measures 
on the measurable space (A, 2^), and denote by SubDisc(X) the set of discrete 
sub-probability measures on the measurable space (A, 2^). We call a discrete 
probability measure a Dirac measure if it is concentrated at a single point, say 
xq, by assigning measure 1 to any set containing xq and measure 0 to any other 
set. In the paper we use sub-probability measures to denote the fact that some- 
times there is no progress from some state s. We view a discrete sub-probability 
measure pi over A as a discrete probability measure over A U {T} where T 
denotes the fact that there is no progress and /x(T) = 1 — /x(A) denotes the 
probability of not making any progress. When /i(A) = 0, then we can think of 
fj, as concentrated at T. For this reason we abuse notation and say that a dis- 
crete sub-probability measure p, over A is Dirac if either /x is a Dirac probability 
measure or /x(T) = 1. We drop the set notation whenever a set is singleton. 

A function / : Ai — >■ X 2 is said to be a measurable function from a measur- 
able space (Ai,.7^i) to a measurable space (A 2 , A 2 ) if for each element C of X 2 , 
G Ai. In such case, given a measure /x on (Ai,Ai), we can define the 
measure induced by /, denoted by /(/x), as f{p){C) = /x(/“^(C')). A random 
variable defined on a measurable space (A, A) is a measurable function from 
(A, A) to where B denotes the cr-field generated by the open intervals 

of the reals, also called the Borel a-field over the reals. 
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3 The Model 

In this section we give an overview of probabilistic automata. We omit the dis- 
tinction between internal and external actions since it is irrelevant for the pur- 
pose of this paper. The reader interested in more details is referred to [11]. 



3.1 Probabilistic Automata 

A probabilistic automaton A is a tuple {S,s^S,T>) where S' is a set of states, 
s G S is a start state, H is a set of actions, and T) C S x E x Disc{S) is a 
transition relation. 

For convenience, states are ranged over by r, q, s; actions are ranged over by 
a, b, c; discrete distributions are ranged over by /i. We denote the elements of a 
probabilistic automaton A by S, s, E, T>, and we propagate primes and indicies. 
Thus, the elements of a probabilistic automaton A' are denoted by S[,s\,E[,V^. 

We call an element of 21 a transition. We say that a transition (s, a, p) is 
enabled from s and is labeled by a. We also say that s enables {s,a,p). We call 
a transition (s, a, p) Dirac if /r is a Dirac measure. For a state s we denote by 
D{s) the set of transitions of V that are enabled from s. 

We call a state s Dirac if all the transitions enabled from s are Dirac; we call 
s deterministic if it enables at most one transition for each action; we call s sin- 
gleton if it enables at most one transition. We say that a probabilistic automaton 
A is Dirac, deterministic, singleton, if each state of A is Dirac, deterministic, 
singleton, respectively. 

For comparison with other models, probabilistic automata can be seen as 
an extension of ordinary automata (also called labeled transition systems) since 
an ordinary automaton is essentially a Dirac probabilistic automaton. The reac- 
tive systems of [5] are deterministic probabilistic automata; thus, probabilistic 
automata could be called alternatively nondeterministic reactive systems. The 
probabilistic automata of [9] are equivalent to probabilistic automata. Markov 
Decision Processes [2] are equivalent to deterministic probabilistic automata. 
The alternating model of [3] can be seen as a probabilistic automaton where 
each state is either Dirac or singleton. 



3.2 Executions 

A potential execution of a probabilistic automaton A is a finite or infinite se- 
quence of alternate states and actions, a = soaiSia 2 S 2 ■ ■ ■, starting from a state 
and, if the sequence is finite, ending with a state. Define the length of a, de- 
noted by I a I, to be the number of occurrences of actions in a. If a is an infinite 
sequence, then |q;| = oo. For a natural number i < jaj, denote by a[i] the state 
Si. In particular, a[0] is the start state of a. If a is finite, then denote by a[T] 
the last state of a. 

Two potential executions a = sooisi • • • a„s„ and a' = Sga^Sj • • • can be 
concatenated if s„ = Sg. In such case the concatenation, denoted by a ' o', is 
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the potential execution sooisi • • • dnSna'is'i ■ ■ • . If a = a\ ' «2, then we say that 
a\ is a prefix of a, denoted by «i < a. 

An execution of a probabilistic automaton A is a potential execution of 
A, a = soaiSia2S2 ■ ■ ■, such that for each i < |a| there exists a transition 
(si_i, Oi, /ii) of V where Hi{si) > 0. An execution a is said to be initial if 
a[0] = s. Denote by execs{A) the set of executions of A and by iexecs{Af) the 
set of initial executions of A. Similarly, denote by execs* {A) and iexecs*{A) the 
set of finite executions and finite initial executions, respectively, of A. 

An execution is the result of resolving both nondeterminism and probability 
in a probabilistic automaton and records the sequence of states that the system 
goes through, along with the sequence of actions it engages in. In a probabilistic 
setting it is useful to know the probability distributions over executions that arise 
after resolving the nondeterminism. This is the argument of the next section. 

3.3 Schedulers 

A scheduler for a probabilistic automaton A is a function a : execs* {A) — >■ 
SubDisc{V) such that for each finite execution a, a{a) £ S'ul?Bisc(T>(a[±])). We 
say that a scheduler is Dirac if it assigns a Dirac measure to each execution. 

In other words, a scheduler is an entity that given a finite execution a chooses 
arbitrarily either to stop or to perform one of the transitions enabled from the 
last state of a, possibly using randomization in its choices. Observe that the 
choice of a scheduler is based on the full history. A scheduler is the entity that 
resolves the nondeterminism in a probabilistic automaton. In the context of 
algorithms a scheduler is usually called an adversary since it is seen as an entity 
that tries to degrade the performance of an algorithm as much as possible (the 
worst case scenario is what matters); in the context of Markov Decision Processes 
an adversary is called a policy since the objective of the area of Markov Decision 
Processes is to find the best possible strategy to improve performance. What 
matters from our point of view is that schedulers, adversaries and policies are 
the same entity, which is the entity that resolves nondeterminism. 

In the literature there is a distinction between probabilistic and deterministic 
schedulers. In this paper we have chosen to use the word scheduler for a proba- 
bilistic scheduler and the word Dirac scheduler for a nondeterministic scheduler. 
The reason for our choice is to avoid overloading the term “nondeterministic” . 

Consider a scheduler a and a finite execution a with last state q. The dis- 
tribution a (a) describes how to move from q. Specifically, one of the transitions 
enabled from q is selected randomly according to a (a); then a target state is 
chosen according to the selected transition. The result of the action of ct is a 
pair {q,fJ,cr{a))j which we call the combined transition according to cr(a), where 
p,a(a) is a distribution of SubDisc{S x S) defined as follows: for each pair (a, s), 

Ma(a)((a,s)) = ^ a{a){{q,a,p))p,{s). 

(g,a,/j,)GX> 

The result of the action of a scheduler from a state s can be represented as 
a Markov process whose states are executions of A with start state s. We call 
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these objects probabilistic executions. Formally, the probabilistic execution of A 
induced by a scheduler <j and a state s is a process {Q, p) where Q is the set of 
executions of A that start with s, and /x is 0 except for pairs of the form {q, qar) 
where it is defined as follows: p,{q,qar) = 

Given a scheduler a and a state s we can define a probability measure 
on the paths of the induced probabilistic execution. The measure is defined 
according to the cone construction, where a cone is a set of the form Ca = {ex' \ 
cx < a'}, a being a finite execution that starts with s, and pa-,s is defined on 
the cr- field generated by the cones. The definition of pa-,s on the cones is given 
inductively as follows: 

1 . ^ 1 ; 

2- pa.siCaaq) — (Gc)^cr(a) ((^5 ^)) ■ 

It is a standard argument [4] to show that pcr,s is cr-additive and that it can be 
extended uniquely to the cr-field generated by the cones. 

Throughout the paper we usually refer to a probabilistic execution as the 
corresponding measure over executions pa,s and we omit either cr or s whenever 
they are not needed or clear from the context. 

4 Coin Lemmas for a Single Experiment 

In this section we give our new formulation of a coin lemma for a single exper- 
iment and we compare it with the formulation of [12]. We start with a simple 
example that illustrates the structure of a coin lemma. 

Example 4- Consider the experiment of rolling a die with 6 faces. We know that 
each face has probability 1/6. If we have a probabilistic automaton where a 
transition encodes the action of rolling a die and where a beep signal is sent 
whenever the outcome of the die is an even number, then we could say that in 
each probabilistic execution the probability of observing a beep signal is 1/2. 
However, it may be the case that a die is not rolled in every probabilistic exe- 
cution; thus, a correct statement would be that in each probabilistic execution 
the probability of either not rolling any die or observing a beep signal is at least 
1/2. This simple observation is at the base of the formulation of a coin lemma. 
Although the statement seems to be obvious, when the number of experiments 
grows the result is not obvious any more, and indeed in several occasions the 
principle was not applied correctly. 



4.1 The Old Formulation of a Coin Lemma 

We formulate a coin lemma for a single experiment as it is formulated in [12]. 

Lemma 1. Let Abe a probabilistic automaton, and let (a, U) be a pair consisting 
of an action a of A and a set of states U of A. Let p be a real number between 
0 and 1 such that for each transition {s,a,p) of A, p{U) >p. 
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For each probabilistic execution of A let FIRST{a,U){pcr) be the set of 
executions a of A such that either a does not occur in a, or a occurs in a and 
the state reached after the first occurrence of a is a state ofU. 

Then, for each probabilistic execution of A, pa{FIRST{a, U){pa)) > P- 

The experiment described by the coin lemma above consists of performing 
a transition labeled by a, and checking whether a state from U is reached after 
performing a. If there are multiple occurrences of a-labeled transitions, then 
we observe the first occurrence. The hypothesis of the lemma requires that in 
each a-labeled transition the probability of reaching a state from U is at least 
p. The conclusion states that the probability of either not performing any a- 
labeled transition or reaching a state from U after performing the first a-labeled 
transition is at least p. It is easy to generalize Lemma 1 to a coin lemma where 
the i-th occurrence of an action a is observed. 

Example 5. Returning to the example of rolling a die, we need a special action, 
say roll, to label all transitions where a die is rolled. The set U is the set of states 
where the die gives an even number, and p is 1/2. 

The rule of Lemma 1 that associates an event with each probabilistic exe- 
cution considers all those executions where either a does not occur or the first 
occurrence of a is followed by a state from U. The coin lemma guarrantees that 
all the events returned by the rule have minimum probability p. If we show that 
some good property holds for each execution of the events returned by the rule, 
then we can conclude that the good property holds with probability at least p 
no matter how the nondeterminism is resolved. In particular we are forced to 
show that either a occurs always or that the good property holds whenever a 
does not occur. 

4.2 The New Formulation of a Coin Lemma 

The old formulation of coin lemmas relies strongly on the labels associated with 
the transitions of a probabilistic automaton. The rule to determine whether an 
experiment takes place (in the case of Lemma 1 the experiment takes place at 
the first occurrence of a) is fixed in the formulation of the coin lemma. Thus, for 
each kind of experiment a new coin lemma should be formulated. Furthermore, it 
is difficult to formulate coin lemmas based on non-trivial methods to determine 
when an experiment takes place (e.g., the first a after three consecutive a’s lead 
to a state from U). 

The fact that success means reaching a fixed set of states U gives us limited 
flexibility as well. An example of limited flexibility can be observed in the proof 
of correctness of the consensus algorithm of Ben Or [1, 11]. The experiment 
consists of flipping a coin and the successful result depends on the past history. 
Sometimes it is successful to obtain head and sometimes it is successful to obtain 
tail. A coin lemma formulated in terms of a unique set of states U does not suffice. 

Rather than formulating ad-hoc coin lemmas each time, here we propose a 
new formulation of coin lemmas that relies on random variables both to identify 
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the places where an experiment takes place and the successful results of an 
experiment. A coin lemma does not fix any more the rule to determine where 
an experiment takes place; rather, it takes the rule as an argument in the form 
of a random variable. 

An experiment either takes place or does not take place. Thus, we can define 
a binary function that associates 0 to those executions where the experiment 
does not take place and 1 to those executions where the experiment takes place. 
We require the binary function to be measurable, thus a random variable. If an 
experiment takes place in a finite execution a, then the experiment takes place 
in any extension of a, since in any extension of a the prefix a has occurred. If 
an experiment takes place in a, then we can identify the exact point at which 
the experiment takes place by looking at the minimum prefix of a where the 
experiment takes place. The important concept for us is that an experiment 
must take place at some finite point. Thus, if an experiment takes place in an 
infinite execution a, then it must take place also at some finite prefix of a. The 
outcome of an experiment can be captured by some other random variable whose 
value is observed only after the experiment takes place. 

We now start to capture formally the notion of an experiment, which consists 
of a random variable that identifies where the experiment takes place, called an 
experiment detector, and another random variable that describes the outcome of 
the experiment. We consider a generic probabilistic execution /i and we denote 
by T the u- field on which /i is defined. 

Definition 1. Let X be a random variable on T . We say that X is finitely 
determined if there exists a root function px such that, px{oc) is a finite prefix of 
a for each execution a, and whenever X{a) 0 for some execution a, X(a') = 
X{a) for each execution a' such that px{oc) <a'<a. 

We say that the random variable X is persistent if, whenever X(a) ^ 0 for 
some execution a, that X(a') = X{a) for each execution a' such that a < a' . 

In other words, the random variable X is finitely determined if, whenever its 
value is not 0 in an infinite execution, we can identify a finite point after which 
its value is fixed; the random variable X is persistent if its value does not change 
once it is different from 0. The root function is not unique in general. Normally 
we can take px{oi) to be the minimum prefix of a that satisfies the property of 
Definition 1. Although we do not state it explicitly in the rest of the paper, we 
always assume that a finitely determined random variable is equipped with its 
root function. 

Definition 2. A random variable is binary if it takes values in the set {0, 1}. An 
experiment detector is a finitely determined persistent binary random variable. 
An experiment is a pair {D, U) of persistent random variables where D is an 
experiment detector. An experiment is said to be finitely determined if U is 
finitely determined as well. We say that an experiment is a binary experiment 
if U is binary. 

Whenever the random variables D and U are binary, we can refer to D and 
U as sets and we can use the classical set notation when referring to D and U . 
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Observe that any measurable boolean condition on any random variable U gives 
rise to a binary variable. Thus, it is always possible to derive binary experiments 
from generic experiments. 

Example 6. In the die example the experiment is a pair {Dd, Ud) where Dd{a) = 
1 whenever the action of rolling the die takes place in a, and Ud{a) = 1 if the 
state reached after the die is rolled corresponds to an even outcome. 

In the example of Lemma 1, the experiment is the pair (Da,Ua) where 
E>a{a) = 1 if a contains an occurrence of action a, while Ua{a) = 1 if a oc- 
curs in a and the state reached in a after the first occurrence of a is a state of 
the set U. 

In the formulation of Lemma 1 there is an hypothesis stating that the prob- 
ability of reaching a state from [/ in a transition labeled by a is at least p. In 
other words, once the experiment takes place, the probability of success must 
be p. We capture this idea with the notion of a p-successful experiment, which 
is expressed in terms of conditional distributions. The root function is used to 
determine the point where the experiment takes place. 

Definition 3. Given a binary experiment (D,U) and a number p € [0,1], we 
say that the experiment {D, U) is p-successful in a probabilistic execution p. if for 
each execution a G D it is the case that either fJ,{Cp(^a)) = 0 or p(Cf |C'p(Q,)) > p. 

The first result about p-successful experiments is that the local condition 
imposed in the definition implies a global condition as well on the probability of 
success whenever the experiment takes place. 

Lemma 2. Let A be a probabilistic automaton, p. be a probabilistic execution 
of A, and {D, U) a p-successful experiment for p. Then, either p{D) = 0 or 
p{U\D)>p. 

Proof. If p{D) = 0 then we are done. Otherwise, let 9 be the set of minimal 
roots given by pu, i.e., the minimal elements in the image under p£> of the 
event {D = 1}. By definition of 9, p{D) = Then, p{U\D) = 

p{D n U)/p{D) = KCo,)p{U\C^)/ p{Ca). By hypothesis, since the 

elements of 9 are roots, for each a G 9 we have p{U\Ca) > p- Thus, p{U\D) > p. 

The global condition proved in Lemma 2 leads directly to our new formulation 
of a coin lemma for single binary experiments. 

Proposition 1. Let A be a probabilistic automaton, p be a probabilistic execu- 
tion, and {D, U) a p-successful experiment for p. Then, p{~'D V U) > p. 

Proof p{^D V [/) = p{^D) + p{D 0 [/) = p{~^D) + p{D)p{U\D) > p{^D) + 
p{D)p > p{^D)p -b p(D)p = p. 
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Example 1. Lemma 1 is just a special case of Proposition 1. To show that the 
experiment (Da,Ua) is p-successful we extend the cone notation by considering 
cones of executions that end with an action as well. Then, the root of an execution 
a that contains action a is a truncated at the first occurrence of a. The condition 
on the a-labelled transitions of Lemma 1 implies directly that (Da,Ua) is p- 
successful. Observe that in any probabilistic execution p the event -'Da V Ua 
coincides with the event FIRST{a,U){p). 

Using the structure of Proposition 1 we can formulate easily coin lemmas for 
the i**' occurrence of an action a or more elaborate coin lemmas that identify 
the occurrence of any generic condition. An example is the coin lemma needed 
in [11] for the analysis of the randomized consensus algorithm of Ben Or [1]. In 
this case the experiment consists of flipping a coin and the successful outcome 
depends on what happened before flipping the coin, which can be represented 
easily with the random variable U . 

5 Properties of Single Experiments 

There are several properties of p-successful experiments that can be studied 
separately and that lead to new coin lemmas. In particular, we can derive gener- 
alizations of the coin lemmas of [12] that identify the first experiment that takes 
place among many. 

Definition 4. A sub -experiment of an experiment {D, U) is an experiment 
{D', U') such that D' C D, po' = po, and U C U' . 

In a sub-experiment we perform the actual experiment in fewer places and impose 
fewer restrictions for its success. The condition on the root function ensures that 
we are not moving the places at which an experiment takes place. 

Proposition 2. Any sub -experiment of a p-successful experiment is p- 
successful. 

Binary experiments can be combined, leading to new binary experiments. 

Proposition 3. Let A be a probabilistic automaton and pt be a probabilistic 
execution of A. Let (C, U) be a p-successful experiment for p, and {D, V) be a 
q-successful experiment for p. Let E = CU D and Z = (C fl [/) U (D fl U) . Then 
{E, Z) is a min{p, q)-successful experiment for p. 

The experiment {E, Z) of Proposition 3 is considered to be successful when- 
ever at least one of the two experiments (C, U) and {D, V) occurs and is suc- 
cessful. By hypothesis we know that when one experiment takes place, then it 
is successful with probability either p or q. Thus, no matter what experiment 
takes place, the probability of success is at least min(p,q). 

Example 8. We can combine sub-experiments and Proposition 3 to observe the 
result of the first experiment among two as follows. Consider two binary ex- 
periments {C,U) and (D,V). Let C < D denote the detector of experiment C 
occurring not after experiment D (experiment D may not occur at all) ; similarly, 
let D < C denote the detector of experiment D occurring before experiment C. 
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Observe that {C < D, U) is a sub-experiment of (C, U) and {D < C, V) is a 
sub-experiment of {D,V). By applying Proposition 3 to (C < D,U) and {D < 
(7, V), we obtain an experiment that is successful whenever the first experiment 
that take place among C, D is successful. 

Thus, the probability that either no experiment takes place or that the first 
experiment that takes place is successful is at least min{p,q). By an inductive 
argument the same construction can be generalized to an arbitrary finite number 
of experiments. 

Example 9. By keeping the setting of Example 8, if we use C to detect the 
first occurrence of an action a, D to detect the first occurrence of an action b, 
U to check whether a state from some set X is reached immediately after the 
occurrence of a, and V to check a state from some set Y is reached immediately 
after the occurrence of b, then, applying Proposition 3 to (C < D, U) and {D < 
C, V), together with Proposition 2, we derive the coin lemmas of [12] that deal 
with the outcome of the first action among two. Such coin lemmas are used in 
the analysis of the randomized dyning philosophers algorithm of Lehman and 
Rabin [6, 7]. 

One last result about binary experiments considers the union of two experi- 
ments with disjoint successful outcomes. In this case, the probabilities of success 
add up as expected. 

Proposition 4. Let A be a probabilistic automaton and p be a probabilistic 
execution of A. Let {D, U) be a p-successful experiment for p and {D, V) be 
a q-successful experiment for p such that U C\V = 0. Then {D,U UP) is a 
p + q-successful experiment. 

6 Multiple Experiments 

A stochastic process consists of a collection of experiments. In this section we 
describe several ways of combining experiments, leading to general formulation 
of coin lemmas. We first need some preliminary definitions to understand when 
two experiments can be treated as independent. 

Definition 5. We say that two experiments (C, U) and {D, V) are separated if 
for each execution a & C C\ D , pc{of) yf pd{cx). 

We say that two finitely determined experiments (C, U) and {D, V) are or- 
dered if they are separated and for each a € C D D either 

— pc{of) < PD{ot) and U{a) yf 0 implies pu{ot) < pd{c(), or 

— pd{oc) < pc{of) and V{a) yf 0 implies pv{of) < Pc{of). 

Informally, two experiments are separated if they take place at different points 
in an execution, and two experiments are ordered if one experiment is com- 
pleted (the outcome is observed) before the other experiment starts. Ordering 
of experiments is necessary to ensure that two experiments are independent. 
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6.1 Multiple Binary Experiments 

Several times we are interested in observing the simultaneous success of a collec- 
tion of experiments. The proposition below, which generalizes the coin lemmas 
about conjunction of [12], shows how to deal with two experiments. The gener- 
alization to any finite number of experiments follows by an inductive argument 
and can be derived from Proposition 6 in the next section. 

Proposition 5. Let A be a probabilistic automaton and p. be a probabilistic 
execution of A. Let (C, U) be a p-successful experiment for p and {D, V) be a 
q-successful experiment for p such that (C.U) and (D,V) are ordered. Then 
p{{-^C\JU)C\{-^D\JV))>pq 

In the coin lemma above we are interested in the outcome of two experi- 
ments. The ordering condition imposes that one experiment terminates before 
the other experiment starts. This condition avoids cases like those where the 
success of an experiment coincides with the failure of the other experiment, 
which would invalidate the lower bound on probabilities. In other words, since 
the experiments are performed at different points in time and their outcome is 
observed independently, the two experiments are guarranteed to be independent. 

6.2 Multiple General Experiments 

Our main objective is to derive a formulation of a coin lemma that is as general 
as possible and that does not depend too much on the underlying stochastic 
process as well as the underlying computational model. In this section we give 
two coin lemmas that are based on possibly countably many experiments whose 
outcomes are required to satisfy some specific property identified by a binary 
measurable function. 

The first coin lemma that we propose deals with a scenario where the ex- 
periments may occur in any order within a probabilistic execution. To accept 
arbitrary orderings of experiments, each occurrence of an experiment is required 
to respect the same probability measure. 

Proposition 6. Let A be a probabilistic automaton and p be a probabilistic 
execution of A. Let {{Di, Xi)}i^j , be a finite or countable collection of pairwise 
ordered experiments with root functions {pi}i^i- Suppose that for each i G L there 
exists a measure pi such that for each execution a G Di, pi = p{-\pi{a)), and 
let V be the measure Xi^jpi, the product of the pi’s as independent measures. 

Let X be f{Xi, X 2 , • • •); where f is a binary measurable function defined on 
the Xi ’s. Let Y be a binary function such that Y = 1 iff there exists a collection 
of real numbers {yi}i^i such that, for each i G I, either yt = Xi or Di = 0, and 
such that f{yi,y 2 , ■■■) = 1. 

Then, p{Y) > v{X). 

Proposition 6 considers a collection of experiments {{Di,Xi)'\i^i. These ex- 
periments are assumed to be pairwise ordered, so that there is no place where 
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two distinct experiments take place at the same time. This ensures indepen- 
dence. Each time an experiment (Di,Xi) takes place there is always the same 
distribution that describes conditional on the specific occurrence of the 
experiment. The distribution v describes the composition of the /ii measures as 
independent measures. 

Function / is a boolean test on the variables Xi’s. Function Y checks whether 
function / has a chance to be successful given the outcome of the experiments 
that take place. In other words, if there is a possibility to fix arbitrarily the 
outcome of the experiments that do not take place so that / evaluates to 1, then 
Y evaluates to 1 as well. For this purpose, observe that the j/'s must coincide 
with the Xi’s whenever the corresponding experiments take place. 

In the coin lemma that follows the experiments are required to occur ac- 
cording to a predetermined order and no experiment may occur if the previous 
experiments have not occurred yet. In this last case we can simply look at the 
global outcome of an experiment without looking at each single occurrence. The 
main difference between the coin lemma below and Proposition 6 is in the con- 
ditions enforced on the measures y^i’s. 

Proposition 7. Let A be a probabilistic automaton and p, be a probabilistic 
execution of A. Let {{Di, Xi)}i^j , be a finite or countable collection of pairwise 
ordered experiments such that Di < Dj whenever i > j. For each i £ L let pi 
be the measure p{-\Di), i.e., the measure of X^ conditional on Di. Let v be the 
measure Xi^jpi. 

Let X be f{Xi, X 2 , ■ ■ ■), where f is a binary measurable function defined on 
the Xi ’s. Let Y be a binary function such that Y = 1 iff there exists a collection 
of real numbers {yi}iei such that, for each i € I, either yi = Xi or Ii = 0, and 
such that f{yi,y 2 , ■■■) = 1- 

Then, p(Y) > i^{X). 

A simple consequence of Proposition 7 is that, for an experiment {D, X) and a 
real number q, /x(-'/V(X > q)) > p{X > q\D). In particular, if p{X > q\D) > p, 
then p{-'D V (X > q)) > p. The result is obtained by considering a function / 
that checks whether X > q. 

Example 10. Consider a finite collection (Di, Xi), • • • (D^, X^) of pairwise sep- 
arated binary experiments, and suppose that each experiment (Di,Xi) is Pi~ 
successful for some probability pi. Let / be the minimum of the X^’s. Then 
the random variable Y of Proposition 6 identifies all the executions of the set 
(-'Z?iUXi)n- • •n(-'DfcUXfc). Furthermore, the lower bound given by Proposition 
6 is Pi • • -pk, thus proving the generalization of Proposition 5. 



Example 11. If we consider a probabilistic automaton where a possibly infinite 
sequence of coin flips occurs and we identify each experiment detector Di with 
the coin flip and Xi with the i**' outcome, then Proposition 6 gives us the 
coin lemma for random walks that appears in [8] . 
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7 Concluding Remarks 

We have reformulated coin lemmas in terms of random variables and conditional 
distributions, thus generalizing all known coin lemmas and making them inde- 
pendent of the specific model of probabilistic automata. An important step is the 
formulation of the notion of p-successful experiments, which are an abstraction 
of the building blocks of the coin lemmas of [12]. The new formulation of coin 
lemmas highlights more precisely the fact that, in order for a system to function 
correctly with high probability, the adversarial scheduler should not be able to 
gain any advantage by avoiding to schedule any random draws. 

The next step in the generalization of coin lemmas is to understand better 
what happens when we deal with expectations. So far the only result about ex- 
pectations appears in [8], where an upper bound on the expected termination 
time for a random walk with barriers is studied within a framework with nonde- 
terminism. The formulation of the coin lemma in [8] is specific to the problem 
under examination and it is not clear yet how to derive such bounds in general. 
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Abstract. This paper presents a modelling language, called MoDeST, 
for describing the behaviour of discrete event systems. The language 
combines conventional programming constructs - such as iteration, al- 
ternatives, atomic statements, and exception handling - with means 
to describe complex systems in a compositional manner. In addition, 
MoDeST incorporates means to describe important phenomena such as 
non-determinism, probabilistic branching, and hard real-time as well as 
soft real-time (i.e., stochastic) aspects. The language is influenced by 
popular and user-friendly specification languages such as Promela, and 
deals with compositionality in a light-weight process-algebra style. Thus, 
MoDeST (i) covers a very broad spectrum of modelling concepts, {ii) 
possesses a rigid, process-algebra style semantics, and (Hi) yet provides 
modern and flexible specification constructs. 



1 Introduction 

System design is primarily focussed on functional aspects. Non-functional as- 
pects such as reliability and performance typically play a role - if at all - in 
the final stages of the design trajectory. To overcome this problem, sometimes 
identified as the insularity problem of performance engineering iniia, it has 
been widely recognised that quantitative system aspects should be considered 
during the entire system design trajectory. Although a complete insight in the 
quantitative aspects might not be present at each design stage, even with partial 
information (or rough estimates) design alternatives may be rejected early due 
to unsatisfactory performance or dependability characteristics. For this purpose, 
modelling techniques used by system engineers or those that provide an easy 
migration path for users need to be adapted to take quantitative system aspects 
into account. 

This has resulted in extensions of light-weight formal notations such as SDL 
and UML on the one hand, and the development of a whole range of more 
rigorous formalisms based on e.g., stochastic process algebras, or appropriate 
extensions of labelled transition systems (such as timed and probabilistic au- 
tomata CEU). Light-weight notations are typically closer to engineering tech- 
niques, but lack a formal semantics; rigorous formalisms do have such formal 
semantics, but their learning curve is typically too steep from a practitioner’s 
perspective. In this paper, we propose a description language that is intended 
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to have a rigid formal basis (i.e., semantics) and incorjwrates several ingredients 
from light-weight notations such as exception handling^, modularisation, atomic 
statements, iteration, and simple data types. The semantics enables formal rea- 
soning and provides a solid basis for the development of tool support whereas 
the light-weight ingredients are intended to pave the migration path towards 
engineers. 

Important rationales behind the development of the description language, 
called MoDeST (Modeling and Description language for Stochastic Timed sys- 
tems), are: 

— Orthogonality. The language has been set up in an orthogonal way such that 
timing and probabilistic aspects can easily be added to (or omitted from) a 
specification if these aspects are of (no) relevance. 

— Usability. Syntax and language constructs have been designed to be close to 
other commonly used languages. The syntax resembles that of the program- 
ming language C and the modelling language Promela EH- Data modulari- 
sation concepts and exception handling mechanisms have been adopted from 
modern object-oriented programming languages such as Java 1E|. Process 
algebraic constructs have been strongly influenced by FSP (Finite State Pro- 
cesses m) a simple, elegant calculus that is aimed at educational purposes. 

— Practical considerations. The design of the language and the development of 
accompanying prototype tool-support have taken place hand-in-hand. Con- 
siderations about the tool handling of language constructs have been a driv- 
ing force behind the language development. 

— Expressiveness. We have identified a handful of semantic concepts which are 
well-established in the context of computer-aided verification and modelling 
formalisms for stochastic discrete event systems: 

(1) Action nondeterminism is often used in concurrent system design to leave 
parts of the description underspecified, and is an appropriate means to 
reflect that the order of events in concurrent executions is out of the 
control of a modeller. 

(2) Probabilistic branching is a way to include quantitative information 
about the likelihood of choice alternatives. This is especially useful to 
model randomized distributed algorithms, but also suitable to represent 
scheduling strategies, quantify data dependencies etc. on an abstract 
level. 

(3) Clocks are a means to represent real time and to specify the dynamics 
of a model in relation to a certain time or time interval, represented by 
a specific value of a clock. 

(4) Delay nondeterminism allows one to leave the precise timing of events 
unspecified. In many cases, the system dynamics depends on events tak- 
ing place in some time interval (e.g., prior to a time-out) where it is left 
unspecified when in the interval the event will occur. 

^ Exception handling in specification languages has received scant attention. Notable 
exceptions are Enhanced-LOTOS and Esterel (3. 
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(5) Random variables are often used to give quantitative information about 
the likelihood of a certain event to happen after or within a certain time 
interval. 



While (1) and (2) affect the dynamics of a model via the (discrete) set of next 
events, (4) and (5) are means to affect the model dynamics by the (continuous) 
elapse of time. Thus, (1) and (4) describe two distinct types of nondeterminism, 
while (2) and (5) represent distinct types of probabilistic behaviour. We believe 
that each of these concepts is indispensable if striving for an integrated consider- 
ation of quantitative system aspects during the entire system design trajectory. 
However, we are not aware of any other formalism, model, or tool that is power- 
ful enough to cover the complete spectrum spanned by this classification. Some 
approaches however come close, among them |29l4llOI7l2b| . We achieve the full 
expressiveness by using a model that integrates timed automata P (using the 
deadline style of 0), stochastic automata [m, and (simple) probabilistic au- 
tomata ISH- These three ingredient models have been selected from a wide range 
of possible alternative models. They were chosen because they complement each 
other very well and yield precisely the desired expressiveness. Due to their in- 
dividual compositional properties, the resulting model is elegant to use in the 
context of a compositonal semantics for the language MoDeST. 

We claim that the language eases the description of a wide range of systems, 
because, in summary, it combines a rigid formal semantics with the following 
key features: 



— light-weight control structures such as iteration, and exception handling 

— simple data types that can be user-defined using modularisation (packages) 

— composition and abstraction mechanisms to structure specifications 

— atomic statements to control the granularity of transitions 

— nondeterministic and probabilistic alternatives 

— nondeterministic and probabilistic timing 



This paper presents the formal syntax and semantics of MoDeST and dis- 
cusses the relationship to existing models for probabilistic systems. The reader 
interested in data and data type treatments in MoDeST is referred to m 



Organisation of the paper. Section Q introduces the language ingredients of 
MoDeST in an incremental way. Section 0 defines the syntax and semantics 
formally. Section 0 discusses the range of models covered by MoDeST. Section El 
briefly addresses some analysis techniques for MoDeST specifications. Finally, 
Section El concludes the paper. For the sake of clarity, this paper focuses on 
behavioural aspects of the semantics and omits considerations on data manipu- 
lation. A full version of this paper is available m 



2 A Gentle Language Primer 

This section introduces the core language features of MoDeST by specifying 
a real-time cashier. This is done in an incremental manner starting from an 
untimed, non-probabilistic description. 
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process Cashier{) { 
do{:: get_prod ;alt{ 

:: cash 
:: set -price ; 
cash } 

} 

} 



The system is informally described as 
follows. In a supermarket customers ar- 
rive at the cashing point and queue in or- 
der to pay their selected products. The 
customers provide their products on a 
conveyor belt and the cashier takes the 
products one-by-one from the belt (this is 
modelled by action get-prod). The prod- 
uct is either cashed (action cash)^ or in case there is no price tag, the cashier 
calls for assistance to establish the price (action set.price) after which cashing 
takes place (action cash). This behaviour is described by the above process, 
where ; denotes sequential execution and :: is used as a separator for the dif- 
ferent alternatives of the choice construct alt . This construct is a way to model 
action nondeterminism. The cashier repeats his (or her) behaviour (indicated by 
do{:: . . .} which is executed repeatedly, unless a break occurs). 



In case more information is available 
about the likelihood with which a cus- 
tomer delivers a product without price 
tag, the nondeterministic choice may be 
replaced by a probabilistic choice. This 
yields the process depicted on the right, 
where weights (in the form of positive re- 
als) are used to determine the likelihood 
with which a certain alternative should be chosen. Here, price information is 
available with probability 0.98 and the price tag is absent with probability 0.02. 
In the terminology of Section lU palt is a means to incorporate probabilistic 
branching. Each probabilistic choice-construct is required to be action guarded, 
i.e., immediately preceded by an action. 



process CashierQ { 
do{:: get -prod palt{ 

:49: cash 
: 1: set-price; 
cash } 

} 

} 



Another uncommon but very ser- 
viceable language construct is the 
possibility to raise and handle excep- 
tions. To illustrate this concept, we 
slightly adapt the description of the 
cashier as depicted on the right. In 
case a product cannot be cashed due 
to an absent price tag, the cashier 
calls for assistance by raising an ex- 
ception (modelled by action no_price 
of exception type) . On handling this 
exception the price is determined and 



process CashierQ { 

do{:: try { get-prod palt{ 

:49: cash 

: 1: throw (no -price) } 

} 

catch no -price { 
set-price; 
cash } 

} 

} 

the product is cashed. 



In a construct like try { P } catch e {Q} the body P in general models the 
normal behaviour, whereas if action e occurs while executing P, an exception is 
raised that shall be handled by Q, i.e., control is passed from P to Q. Note that 
compared to our previous specification, an additional action (of exception type) 
has been introduced to signal the occurrence of the exceptional situation. 
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So far, our descriptions were time- 
less, i.e., we did not include any tim- 
ing considerations with respect to 
the activities involved. In the next 
step, we will put some simple tim- 
ing constraints on the cashier. Like 
in timed automata P, the elapse 
of time in MoDeST is modelled by 
means of clock variables. Values of 
clock variables increase linearly as 
time progresses. For instance, in or- 
der to impose a delay of at least 120 
time units between catching the ex- 
ception no-price and determining the price of the product at hand (set-price), 
we equip the previous description with clock variable y, and obtain the process 
on the right. Clock y is reset just after catching the exception nojprice and the 
price can be determined at any time point after a delay of at least 120 time- 
units as indicated by the when-clause. In fact, each action needs to be preceded 
by a when() constraint, but unless otherwise specified when(true) is a default 
constraint (that can be omitted). 

When-clauses thus indicate when a 
certain action may (i.e. is allowed to) 
happen. Similar to location invari- 
ants in safety timed automata m 
and deadlines in timed automata 
with deadlines [0| , we need a separate 
mechanism to force certain actions to 
happen at some time instant. To that 
end, we use deadlines. For instance, 
the process on the right specifies that 
set-price is enabled from 120 time 
units after catching the exception (as 
before), and that it should happen 
before 240 time units after the catch 
- as indicated by the urgent-clause. More precisely, if the exception is catched 
at time t, say, then setjprice will happen at some time instant t-\-A where A is 
nondeterministically chosen from the closed interval [120,240]. Thus, differences 
in guards and deadline constraints induce delay nondeterminism. 

In general, if an action is guarded by urgent(B), for boolean expression B, 
it must be executed as soon as B becomes true. Therefore, a system is allowed 
to idle as long as none of its activities becomes urgent. The language user can 
influence whether by convention activities are assumed to be urgent (guarded 
by urgent(true)), or non-urgent (guarded by urgent(false)), via setting a flag 
in the preamble of a MoDeST specification. 



process CashierQ { 

do { try { get -prod palt { 

:49: cash 

: 1: throw(no_pnce) } 

} 

catch no -price { 
y = 0; 

urgent(y > 240) 
when(y > 120) 
set-price; 
cash } 

} 

} 



process CashierQ { 

do{:: try { get -prod palt{ 

:49: cash 

: 1: throw(no-price) } 

} 

catch no -price { 
y = 0; 

when(y > 120) 

set-price; 
cash } 

} 

} 
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As a next step, we impose a delay 
on the cashing of the cashier, i.e., 
on action cash. Depending on (the 
price of) the product, environmen- 
tal circumstances (such as the mood 
of the cashier, the time of the day), 
and so on, the duration of cashing 
may vary. We assume that cashing 
takes between 10 and 20 time-units. 
If no more information is available 
this could be modelled in a simi- 
lar way as we just treated set^price. 
However, we now assume that the 
duration of cashing is uniformly dis- 
tributed over the interval [10,20]. In 
this case, the modelling as just above 
does not suffice, as it would choose 
a time instant nondeterministically 
without taking the likelihoods into 
account. To that end, we equip the 
specification with a clock variable x, 



process CashierQ { 

doi:: \ry i get.prod paltj 
:49: CashingQ 
: 1: urgent(true) 

throw(no-price) } 

} 

catch no_price { 

y = 0; 

urgent(y > 240) 
when(y > 120) 
set-price; 
CashingQ } 



} 



} 



process CashingQ { 

[xd=U[10,20] , a; = 0]; 
urgent(a; > xd) 
when(a: > xd) 
cash 

} 



say, and add a float variable xd, say, 
that is used to store a sample value drawn from a probability distribution. Thus, 
the occurrences of cash in process Cashier is replaced by invoking a process 
Cashing depicted on the right. In the latter, the statement [. . .] contains a set 
of assignments that are executed atomically, i.e., without interference with ex- 
ecutions of other processes in the system. In this example, the variable xd is 
assigned a (float) value according to a uniform distribution on interval [10,20], 
and clock x is reset. The urgent- and when-clause make sure that cash takes 
place as soon as x has reached the value xd. 



exception no -price-, 
clock X, y, 
float xd; 

patient yet-prod, cash, set-pricc. 



par{ 



The overall system could be modelled 
by, for instance, the expression on the 
right, where N is the parameter (i.e., the 
length) of the queue. Variables do not 
need to be declared globally, a variable 
(or action, or exception) can equally well 
be declared local to a process. Processes 
are put in parallel via the par{::. . .} con- 
struct. These processes execute their ac- 
tivities independently from each other, 
except that common (non-local) actions 
need to be executed synchronously, a la CSP m- One of the keywords ap- 
pearing in the preamble needs further explanation. We distinguish patient and 
impatient actions. If a patient action is common to multiple processes, then the 
synchronized action becomes urgent as soon as all partners require urgency. In 



} 



ArrivalsQ ; 

Queue{N) 

CashierQ 
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contrast, a process that intends to synchronise on an impatient action is not 
willing to wait for the partner. Thus a synchronized impatient action is urgent 
as soon as at least one synchronization partner requires urgency. 



3 Formal Definition of MoDeST 

This section formally defines the language MoDeST, the underlying operational 
model, and the operational semantics of MoDeST. The semantics maps each 
MoDeST specification on some stochastic timed automata (STA, for short). STA 
combine the power of timed automata Q using the deadline style of |^, stochastic 
automata jldlllj . and (simple) probabilistic automata Before discussing 
syntax and semantics of MoDeST we introduce STA together with other relevant 
concepts. 



3.1 Stochastic Timed Automata 

A probability space is a tuple (17,.?^, P) where fl is the sample space^ T is a, a- 
algebra containing subsets of 12, and P is a probability measure on the measurable 
space (12, .7^). If P is a probability space, we write T-p and P-p for its sample 
space, cr-algebra, and probability measure, respectiveljfl. Let Prob{H) denote the 
set of probability spaces (12, P, P) such that H C H. 

Let Var be a set of typed variables with a distinguished subset Ck C Var of 
clock variables (variables of type clock ). Let RVar be a (finite) set of random 
variables such that RVarflVar = 0. Let Exp be a set of expressions with variables 
in VarU RVar. Let BExp C Exp be the set of boolean expressions, ranged over by 
d, d', g, g' .... A boolean expression is required not to contain random variables. 
A : Var — ^ Exp, is called an assignment. Let Assign denote the set of assignments. 
Let Act be a set of action names. We use a to range over elements of Act. 

Definition 1. A stochastic timed automaton (STA) is a triple {S,Act,—*~), 
where S is a set of locations and — ► C 5x Actx BExpx B Exp xProb( Assign x 5). 

For (s, a, g, d, V) G — we write s P and require that P is a discrete 

probability space. We call g the guard and d the deadline. Intuitively, the system 
is allowed to execute an edge s P whenever it is at location s and the 

guard g holds under the current values of the variables. If in addition the deadline 
d holds, then the system is obliged to execute the edge before time progresses. 
Due to this fact, the system is allowed to wait in location s as long as no deadline 
in one of its outgoing edges becomes true. Once the edge s p jg executed, 

the system moves to location s' assigning values according to A with probability 

Pp((s',2l)). 

We assume familiarity with the basics of probability and measure theory (see 

e.g. E3). 



2 
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Depicted on the right is an example 
STA corresponding to the final Cashier 
specification of Section |21 Locations are 
represented by circles. A probabilistic 
edge is represented by a solid line from 
which doted arrows fan out. The solid 
line is labelled by the guard, deadline, 
and synchronisation label. Each dotted 
arrow represents a probabilistic alter- 
native, and are labelled with a prob- 
ability value and a set of assignments. 

Their target is the next location. Dead- 
lines are prefixed by a ‘u’ (urgent) and omitted if they are false, and guards by a 
‘w’ (when) and omitted whenever they are true. Trivial probabilities and empty 
assignments are also omitted. 

STA provide a symbolic framework to represent stochastic timed behaviour, 
but this representation is too abstract to represent the concrete evolution as 
describe above, which is needed for different kinds of analysis, such as prob- 
abilistic model checking, or discrete event simulation. Therefore, STA have an 
interpretation in terms of timed continuous probabilistic transition systems. This 
interpretation is given in m. 






get.prod 



u(a: > xd) 
w(x > xd) 
cash 



0.98 



0.02 



O' — 

u(true) 

tau 



O 



6 



xd := t/[10,2(|l] 

U{y> 240) 
w(y > 120) 
set_price 



u(true) 

no .price 



u(true) 

tau 

y.= 0 



3.2 Syntax 

In the following we discuss the language constructs of MoDeST. We assume that 
the set of actions Act consists disjointly of: 

— a set PAct of •patient actions, 

— a set lAct of impatient actions, 

— a set Excep of exception names, 

— an action T indicating an unhandled error, 

— an action break indicating the breaking of a loop, and 

— an action tau indicating an unobservable activity called silent step. 

The set of processes of the language MoDeST is given by the following grammar. 



P ::= 



Stop 


error ProcName(ei, . . .,Ck) 


when(6) P 


urgent(6) P alt{::Pi.. 


. ::Pfcj 


act 


1 act palt {-.wi'.asgnj^; Pi . . 


■Wk'.asgn^; P^j 


throw(ea;cp) 


try{P} catch excpi {Pi} . 


. . catch excpp (Pfel 


break 


1 d0|::Pi ... ::Pfe} 




Pi; P2 


1 par{::Pi ... ::Pfc| 




hidejacti, . . 


,actk}P extendjacfi. 


■ ., actk} P 


relabel {acti 


, . . ., actk} by {act'i, . . ., act},} P 
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where, for 1 < i < k, Wi is a, positive integer representing a weight, act, act[ G 
PAct U lAct U {tau}, acti G PAct U lAct, excp, excp^^ G Excep, b G BExp, a G Exp 
not containing random variables, and asgn^ is a list of assignments of the form 
[x\ = ei, X 2 = C 2 , • ■ • , = e„]. A MoDeST process is defined by 

process ProcName{ti xi, . . .,tk Xk) {del P} 

where {x\, . . .,Xk\ G Var, {ti, . . .,tfc} are valid types, del is a sequence of dec- 
larations possibly including process definitions, ProcName is a process name 
and P is as before. We write process ProcName{xi, . . .,Xk) {P} instead in the 
remainder of this paper, for convenience. 

Each set [a;i = ei, . . ., = e„] induces a unique assignment A G Assign de- 

fined by A{xi) = Ci, for 1 < z < n, and A{y) = y ii y ^ {x\, . . .,x„}. There- 
fore, we use [xi = ei, . . ., a;„ = e„] G Assign to refer to its induced assignment 
A G Assign. 

MoDeST provides some further useful operations which are shorthand nota- 
tions for some common constructions. They are described in Appendix El 



3.3 Semantics 

The operational semantics of MoDeST is defined in terms of the stochastic 
timed automaton (5,Act,— ►) where the set of locations S is defined by the 
set of MoDeST processes extended with a special termination mark The 
relation — is defined in the remainder of this section. In the following we 
use Trv(r) to denote the trivial probability space with sample space {r}. We 
also resort to measurable functions. Recall that M : l7i — >■ L ?2 is measur- 
able if M“^(C') G Pi for all C G P 2 and that it induces a probability space 
M(f7i, iFi, Pi) = (J72;-T2,Pi o M“^). In our case, all measurable functions are 
defined to be surjective. Under this condition J72 = M(i7i). 

Primitive operators. Stop does not perform any activity and as such it does not 
produce any transition, act performs action act with no restriction and then 
terminates, break, used to break a do loop, can perform action break with no 
restriction and then terminates, error is a process that indicates an unhandled 
error by persistent executions of action _L. The last of the basic operations, 
throw(ea;cp), raises an exception by executing action excp G Excep. If it is not 
handled, the system ends up in an unhandled error. In all these cases, urgency 
of the execution depends on a global boolean variable urge which can be set to 
true or false in the preamble section of the specification. If set to true, the 
specified system responds to maximal progress (default is false). We get: 

act Trv(V) error Trv(error) 

break, true, urge ^ excp,true,urge ^ Trv(error) 
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Probabilistic prefix, act palt {:wi:asgrii; Pi . . . ■.Wk'.asgn^] performs action 
act with no restriction, but as urgently as indicated by urge. Simultaneously, it 
randomly selects an alternative i € {1, . . ., fc} according to the weights Wi, . . Wk, 
performs an assignment according to asgn^, and continues executing Pi. 

act palt {-.wi'.asgni] P\ . . . -.Wk'.asgnf.] Pk} P 



where P is a discrete probability space with f2-p = {{asgrii, Pi) | 1 < i < A:} and 

def • #{j I 1 < J < A: A asgn^ = asgrij^Pi = Pj} 

Pv{{asgni,Pi)) = — ^ ^ 

Ei=i w, 



Conditions, when (6) P restricts the next activity of P to be performed whenever 
b holds. urgent(6) P enforces P to be urgent whenever b holds: 

p ^ p p a,g,d ^ p 

when(&) P V urgent(6) P p 



Choice. alt{::Pi . . . ::Pfe} executes precisely one Pi, selected in a nondetermin- 
istic fashion: 

p^ a,g,d ^ P^ (1 < t < A;) 

alt{::Pi ... ::P 4 ^b£di^p^ 

Loop. do{::Pi . . . ::Pfc} repeatedly chooses a nondeterministic alternative. The 
execution finishes when one of the processes executes a break. We define the 
semantics of do in terms of alt and an auxiliary operator auxdo: 

do{::Pi ... ::Pfc} auxdo{alt{::Pi . . . ::Pfc}}{alt{::Pi . . . ::Pfc}} 

The semantics of auxdo is given by: 

P -JLdtU- V (a yf break) P V 

auxdo{P}{g} Mdo(P) auxdo{P}{g} V 

where Mdo((^, P')) (A, auxdo{P'}{g}), if P' yf \/, and otherwise, 

Mdo((A,V)) (A,auxdo{g}{g}). 



Exception handling. The process try{P} catch excpi {Pi} . . . catch excpf. {Pk} 
executes P and terminates if P terminates without raising any exception be- 
forehand. If instead P raises an exception excpi, it is handled by executing the 
respective process Pp. 

p a,g,d ^ p ^ {excpi , . . ., excp jf}) 

tryjp} catch excpi (Pij . . . catch excp^. {Pk} Mtry(P) 

p excp„g,d ^ p {l<i<k) 

tryjP} catch excpi (Pij . . . catch excpk {Pk} Trv(P,) 
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Table 1. Alphabet of a MoDeST term 

a(stop) = a(error) = a(break) = a(throw(ea;cp)) = 0 
a{act) = {act} — {tau} 

a{act pall {:wi:asgn^-, P\ . . . -.Wk-asgUf.-, P^}) = a{act) U (JiLi 
a(when(6) P) = a(urgent(fo) P) = a{P) 

a(alt{::Pi . . . ::Pfe}) = a(do{::Pi . . . ::Pfc}) = a(par{::Pi . . . ::P4) = (Jti 
q(Pi; P 2 ) = a(Pi) U a(P 2 ) 

a(try{P} catch excp^ {Pi} . . . catch excp^ {Pfc}) = «(P) u a(Pi) 

a(hide{acti, . . actk} P) = a(P) — {acti, . . actk} 

Q(relabel [acti, . . actk} by {act}, . . act}} P) = 

a(P)[acti/act'i, . . actk/ act}] — {tauj 

a(extend{acti, . . actk} P) — cc{P) u [acti, . . actk} 

a(ProcName(ei, . . Ck)) = a(P) provided process ProcName(xi, . . Xk) |P} 



where Mtry((A,P')) = (A, tryjP'} catch ea;cpi (Pij . . . catch ea:cpj. {Pfc}), if 
P' y, and M,iy((A, V)) =' {A, V). 

Sequential eomposition. Pi; P 2 executes Pi until it finishes. Then it continues 
with the execution of P 2 : 

j, 

Pi; P2 

where M;((A,P')) (A, P'; P 2 ), if P' V, and M;((A, V)) {A^P^). 



Parallel eomposition. par|::Pi ... ::Pfc| executes processes Pi,..., Pfc concur- 
rently, synchronising them on the intersected alphabet, therefore allowing multi- 
way synchronisation. The alphabet of a process P is the set a(P) C PActUlAct of 
all actions P recognises. It is formally defined in Tabled To define the semantics 
of MoDeST parallel composition, we resort to the auxiliary operator ||^, with 
B C PAct U I Act, that behaves like CSP or LOTOS parallel composition |20I5| . 
Thus, par is defined by 

par(::Pi ... ::Pfc} (• ■ -((Pi |Ibi ^^2) |Ib, P3). • ■) IIb,_, 



with Bj = ct{Pi)) O a(Py+i). The behaviour of ||g is formally defined by 

the following rules (we omit the symmetric rule of interleaving) : 



Pi P (a (/ B) 

a,g,d 



,di ^ p^ 



P2 P2 (a e B) 



Pi\\bP2 



^MparP,(P) Pi|IbP2 



a,giAg2,diOd2 



*- Mpar(Pi X P 2 ) 
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where di<>d2 = Ac?2 if a G PAct (that is, if the synchronising action is patient) 
and diOd2 = di V ^2 otherwise (impatient). The operator x denotes the usual 

def 

product on probabilistic spaces, and MparPj ((A, P')) = {A, P' \ \q P2), HP' ^ ^ 
or P2 yf otherwise Mpar^((A, ,/)) {A, yj). Furthermore, 



Mpar((Ai,P(),(A2,P^)) 



def 



if AiU A2 is not a function then 
(0, throw inconsistency) 



< else 

(Al U A2, P[ \\s P^) itPi^VorP^^V 

(AiUA2,V) itPi=P^ = V 



Some remarks are in order. A parallel composition terminates whenever all its 
components terminate. Moreover, notice that the difference between synchroni- 
sation of patient and impatience actions is only given by the way the deadlines 
are related. Since a process that wants to synchronise on a patient action always 
waits for its partner to be ready, then its deadline needs to be relaxed to the 
requirements of the partner. As a consequence, a deadline in a patient synchro- 
nisation is met whenever all the components meet their respective deadlines. 
Instead, a process that intends to synchronise on an impatient action is not 
willing to wait for the partner. Therefore, a deadline in an impatient synchroni- 
sation should be met as soon as one of the one of the synchronising components 
meets its deadlines. Finally, remark that during synchronisation an inconsistency 
of assignments may arise due to different write accesses to the same variable, 
i.e., if Ai{x) ^ A2 {x) for some variable x. We treat this situation by raising a 
predefined exception. 



Relabelling and hiding, relabel {acti, ..., actfe} by {acf'i, ..., act).} P behaves 
like P except that every action acti is renamed by the corresponding act): 

p a,g,d ^ jy f = [acti/ act'^, . . actk/ act').] 
relabel {acti, . . ., act^j by {act}, . . ., act}} P Mreiabel('^) 

where Mreiabei((^j P')) relabel (acti, . . ., actk} by {act}, . . ., act}} P'), if 

P' ^ y, otherwise Mreiabei((A, y/)) (A, y/). 

Hiding is a particular form of relabeling in which actions are renamed by the 
silent action tau. Therefore we define: 

def 

hide{acti, . . ., actfe} P = relabel {acti, ..., actfc} by {tau, ..., tau} P 

k times 

Alphabet extension, extend is only used to extend the alphabet that a process 
recognises (see Table 0 . Otherwise, it does not affect the behaviour: 

p p 

a,g,d 



extend{acti, . . ., actk} P 



Mextend(^) 
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where Mextend((^, ^*0) = {A,extend{acti, . . actk} P') , and 

Mextend((A,V)) =' (AV)- 



Process instantiation. Provided process ProcName{xi, . . .,Xk) {P} is part of 
the MoDeST specification under consideration, ProcName{ei, . . .,€k) behaves 
like P where variables Xi, . . .,Xk are substituted by their respective instantiations 

ei, . . Cfe. 



P[a:i/ei, ■ . .,a;fc/efc] p 

ProcName(ei, . . Cfc) 7? 



provided process ProcName{xi, . . Xk){P} 



In summary, the relation — is the least relation satisfying the above rules. 
The reader is invited to check that the STA depicted in Section Id. 1 1 is derived 
from the final Cashier specification of Section El using these semantic rules (see 
Appendix E] for the shorthand notations used). 



4 Derivable Models 



MoDeST is expressive enough to cover a wide range of timed, probabilistic, non- 
deterministic, and stochastic models. These submodels play a crucial role in the 
context of analysing MoDeST specifications. Table El lists a range of prominent 
models and makes precise which semantic concepts (cf. Section Q each of them 
shares with STA. 

LTS: Labelled transition systems are the basic models of concurrency, they are 
usually analysed with techniques such as model checking or equivalence check- 
ing. They arise from MoDeST by disallowing the use of all time and stochastic 
concepts. 

PTS: Probabilistic transition systems are labelled transition systems where some 
state changes are governed by discrete probability distributions while others are 
nondeterministic. They can be analysed with techniques from Markov decision 
theory, model checking, and equivalence checking |,'S 1 19] . MoDeST subsumes (sim- 
ple) PTS via the palt construct which is action guarded by default. 



Table 2. Submodels of STA 





LTS PTS TA PTA 


MC 


GSMP 


IMG 


SA 


STA 


probabilistic branching 


NO 


YES 


NO 


YES 


YES 


YES 


YES 


YES 


YES 


clocks 


NO 


NO 


YES 


YES 


RESTRICTED 


YES 


RESTRICTED YES 


YES 


random variables 


NO 


NO 


NO 


NO 


EXP. DIST. 


YES 


EXP. DIST. 


YES 


YES 


delay nondeterminism 


NO 


NO 


YES 


YES 


NO 


NO 


NO 


NO 


YES 


action nondeterminism 


YES 


YES 


YES 


YES 


NO 


NO 


YES 


YES 


YES 
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TA: Timed automata are transition systems incorporating an explicit notion 
of real time, represented by continuously moving clocks. Reachability analysis 
and model checking are the usual techniques employed for TA Timed 

automata (with deadlines) arise from MoDeST by abstaining from the use of 
random variables and palt. 

PTA: Probabilistic timed automata are integrating TA and PTS, thus they 
arise from STA if random variables are unused. Reachability analysis and model 
checking have been proposed for PTA 

MC: Continuous time Markov chains are a standard model in contemporary per- 
formance evaluation. An MC is stochastic process where each delay is governed 
by some exponential distributed random variable. Analysis techniques for MCs 
range from the numerical computation of transient and steady state probabili- 
ties to approximate model checking MoDeST allows one to model MC by 

using clocks and exponentially distributed random variables, but in a restricted 
form (guards are right-continuous and clocks can be uniquely mapped on the 
random variables they use). Action and delay nondeterminism is not allowed. 
The model is not closed w.r.t. the operators of the language, e.g. the parallel 
composition of two MCs is not necessarily a MC (but an IMC). 

IMC: Interactive Markov chains are MCs where action nondeterminism can oc- 
cur. Therefore the model is closed w.r.t. the operators of MoDeST. An IMC can 
be analysed with algorithms developed for continuous time Markov decision pro- 
cesses |Sn| , or sometimes be reduced to a MC by factoring the model with respect 
to a weak equivalence m- As with MCs these models can be reconstructed from 
a given STA, if the latter obeys certain restrictions. MoDeST provides shorthand 
notations making it possible to ensure these restrictions by default: A specifi- 
cation where stochastic aspects only make use of these shorthands possesses a 
direct semantics in terms of IMC (without reconstructing the latter from the 
STA semantics). 

GSMP: Generalized semi-Markov processes are a general purpose performance 
evaluation model. Theses stochastic processes are usually analysed using dis- 
crete event simulation, but in specific cases a numerical analysis is also feasible. 
GSMPs arise from MoDeST specifications if action and delay nondeterminism 
does not occur. The model is not closed w.r.t. the operators of the language, 
e.g. the parallel composition of two GSMPs is not necessarily an GSMP (but a 
SA). 

SA: Stochastic automata are basically GSMPs with action nondeterminism 
(hence they are closed under composition), but can also be seen as TA where 
delay nondeterminism is replaced by random variables governing the delays uni 
m As with IMC, specific shorthands can be used to ensure the restrictions 
required to obtain a SA. For instance if X is a random variable then wait(X) 
is an abbreviation for [x = X ]c=Q] urgent(c > x) when(c > x) tau where c 
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(respectively x) is a clock variable (float variable) private to X. Again, these 
shorthands are used to map the MoDeST speciflcation directly on the SA which 
otherwise is retrievable from the STA semantics. 

It is important to remark that the presence of each listed semantic concept - 
apart from action nondeterminism - can be detected syntactically, while parsing 
a speciflcation. This is trivial for probabilistic branching (palt), and obvious for 
clocks, because they have to be declared before use in MoDeST. Use of random 
variables is easily detected while parsing because (exponential or general) con- 
tinuous probability distributions are provided via a predefined class (i.e., type). 
Delay nondeterminism is absent in a speciflcation if for each action the guard and 
deadline agree. So, Table 0 also gives sufficient syntactic criteria for identifying 
submodels while parsing a MoDeST specification. 

Action nondeterminism is a principal feature for compositional formalisms, yet 
it induces that MCs and GSMPs are not closed under composition in general. 
Action nondeterminism can in principle be excluded syntactically by disallowing 
alt and par, but the resulting language is too meager to be of much use. More 
liberal syntactic conditions for absence of action nondeterminism can be adopted 
from |2S|- 

5 Model Analysis 

The identification of well-studied submodels is of crucial practical relevance, 
because the enormous expressiveness of MoDeST comes with the drawback that 
the underlying general model is not well investigated: So far analysis methods for 
the general STA model have not been devised, and their development is ongoing 
work. The general idea behind this work is strongly based on the identification 
of submodels of STA for which analysis methods have been published. Based on 
this knowledge, four different strands can be pursued: 

— Isolate syntactic subclasses of MoDeST that map on well-investigated sub- 
models. As long as the user of MoDeST adheres to such a subset, the proper 
analysis engine can be determined mechanically. 

— Define abstractions from STA to less specific models. One such abstraction 

is to mask the distributions of random clocks i.e., to consider random 
clocks as delay nondeterministic clocks. In this way, any STA can be turned 
into a TA by abstracting the stochastic behaviour. Real-time model checking 
on this TA is safe w.r.t. to the original STA model. 

— Define concretisations from more general models to more specific models. 
This usually means to add additional explicit modelling assumptions, such 
as to assume a particular scheduler to resolve action nondeterminism, or to 
assume that all random clocks follow an exponential or phase-type distribu- 
tion. Note that the quantitative error introduced by such an assumption can 
be unbounded in certain circumstances. 

— Extend or combine analysis methods from submodels of STA to full STA. In 
particular we are planning to integrate real-time model checking of TA with 
numerical recipes for GSMPs. 
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6 Conclusion 

In this paper we have introduced a modelling and description language for 
stochastic timed systems. We have formally defined syntax and semantics of 
MoDeST, and have put the language in the context of other well-studied mod- 
els. The focus of this paper has been the behavioural part of MoDeST. The data 
part is described in In a nutshell, we allow simple and structured data 
types, and modularization (packages). Object-oriented enhancements (classes, 
sub-typing, polymorphism) are under development. 

We are currently implementing a tool suite to support modeling and analysis 
with MoDeST. The language parser is being finalised, and we are working on 
the state space generator now. The main strategy we pursue in this respect is to 
bridge to state-of-the-art verification and analysis tools on the level of the STA 
model. More concretely, we are busy with linking to Uppaal for real-time 
model checking and to Mobius (S| for discrete event simulation and numerical 
analysis. 

Acknowledgement The authors are grateful to Ed Brinksma for inspiring dis- 
cussions. This work is supported by grant TES-4999 of the Dutch Technology 
Foundation (STW) and grant 612.069.001 of the Netherlands Organisation of 
Scientific Research (NWO). 

A Further MoDeST Expressions 

MoDeST provides operations which are shorthands for some common construc- 
tions. For instance, both alt and do allow an else alternative (as in Promela). 
else is a shorthand that can be calculated at compile time, e.g., 

alt{::when(&i) Pi . . . ::when(6fc) ::else Q} 

='' alt{::when(6i) Pi .. . ::when(6fc) Pk ::when(^ VjLi Q}- 

In a probabilistic alternative, either assignments or processes (but not both) 
can be omitted, e.g., act palt {:!: [y = 3] :2: PJV(4) } should be interpreted as 
act palt {:!: [y = 3] :2: [ ] PJV(4) }. Notice however that, strictly speaking, 

the last process is not a legal MoDeST expression since y/ is not in the language. 
The following shorthands for assignment are also allowed in MoDeST: 

[xi = ei, ... , Xk = Ck] urgent(true) tau palt {: 1 : [xi = ei, . . . , = e*] V} 

def r 

X = e = [x = e 

Furthermore, invariants like in safety timed automata m can be defined by 

invariant(5)P urgent(-'5)when(6)P. 

MoDeST also provides other useful forms of relabelling apart from relabel and 
hide, and standard programming constructs are provided, such as: 

while(&){P} do{::when(6) P ::else break}. 
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Abstract. We present and analyze a new probabilistic method for au- 
tomata based LTL model checking of non-probabilistic systems with in- 
tention to reduce memory requirements. The main idea of our approach 
is to use randomness to decide which of the needed information (vis- 
ited states) should be stored during a computation and which could be 
omitted. We propose two strategies of probabilistic storing of states. The 
algorithm never errs, i.e. it always delivers correct results. On the other 
hand the computation time can increase. The method has been embed- 
ded into the SPIN model checker and a series of experiments has been 
performed. The results confirm that randomization can help to increase 
the applicability of model checkers in practice. 



1 Introduction 

Model checking is one of the major recent success stories of theoretical computer 
science. Model checkers are tools which take a description of a system and a prop- 
erty and automatically check whether the system satisfies the property. There 
are now many different varieties of model checkers including model checkers for 
real-time systems and probabilistic systems. 

Practical application of model checking in the hardware verification became a 
routine. Many companies in the hardware industry use model checkers to ensure 
the quality of their products. With the debugging potential afforded by model 
checking, design of hardware components can be made much more reliable and 
moreover model checking is seen to accelerate the design process, significantly 
decreasing the time to market. However, the situation in software model checking 
is completely different. Software is much more complicated system due to its 
size and dynamic nature. To achieve similar benefits as in hardware verification, 
additional methods and techniques need to be explored. 

One of the very successful techniques is randomization. The term “proba- 
bilistic model checking” (or “probabilistic verification” ) refers to a wide range of 
techniques. There are two ways in which probability features in this area. The 
first approach concerns applying model checking to systems which inherently 
include probabilistic information ITTITOI . The second approach concerns sys- 
tems which are non-probabilistic, but of size which makes exhaustive checking 

* This work has been partially supported by the Grant Agency of Czech Republic 
grants No. 201/00/1023 and 201/00/0400. 
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impractical or infeasible PHI. The aim is to use randomization to make model 
checking more efficient, albeit at a cost of establishing satisfaction with high 
probability, possibly with a one-sided error, rather than certainty, or at a cost 
of other resources. While the topic of verification of probabilistic systems has 
been intensively studied, there are only a few attempts to use randomization in 
verification of non-probabilistic systems. 

In the paper we focus on automata based LTL model checking of non-proba- 
bilistic systems. Our aim is to attack the state-explosion problem (the number 
of reachable states grows exponentially in the number of concurrent components 
and is the main limitation in practical applications of model checkers). Various 
techniques and heuristics reducing the random access memory required have 
been proposed. One possible solution (called on-the-fly model checking) is to 
generate only the part of the state graph required to validate or disprove the 
given property. On-the-fly algorithms generate the state space in a depth- first 
manner and keep only track of reached states to avoid doing unnecessary work. 
Another solution makes use of the fact that one of the reasons of the state explo- 
sion problem is the generation of all interleavings of independent transitions in 
different concurrent components. Partial order reduction techniques were intro- 
duced to ensure that many of these unnecessary interleavings are not explored 
during state generation. 

If we have some knowledge about the structure of the state graph in advance 
(before starting the actual verification), we can apply even more efficient heuris- 
tics. As in general it is not the case we suggest to use a probabilistic method 
which can be viewed as a probability distribution on a set of deterministic tech- 
niques. We explore two probabilistic approaches to achieve significant space 
reduction in the depth first search based model checking of non-probabilistic 
systems. 

The core of the first approach is to use randomness to decide which of the 
needed information (visited states) should be stored during a computation and 
which could be omitted. Consequently, the time complexity of the computation 
can increase. The second method simply implements the idea of randomizing 
the branching structure. Both methods are of Las Vegas type, i.e. they always 
deliver the correct answer. In the paper we focus on the first method and report 
on the second one briefly. We stress that both methods are compatible (can 
be used simultaneously) with on-the-fly and partial order reduction techniques. 
We have implemented both methods and the experiments gave surprisingly very 
good results in competition with non-probabilistic approaches. 

The paper is organized as follows. We first review some background on model 
checking using automata, define the corresponding graph-theoretic problem, and 
briefly discuss possible sources for applying randomization. Then we propose the 
probabilistic reduction algorithm and report experimental results achieved. We 
conclude with the description of the second method and with some final remarks. 
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2 Problem Setting 



We consider the following verification problem. A finite state transition graph 
(also called a Kripke structure) is used to represent the behavior of a given 
system and a linear temporal logic (LTL) formula is used to express the desired 
property of the system. The basic idea of automat a-based LTL model checking 
is to associate with each LTL formula a Biichi automaton that accepts exactly 
all the computations that satisfy the formula. If we consider a Kripke structure 
to be a Biichi automaton as well, then the model checking can be described as 
a language containment problem and consequently as a non-emptiness problem 
of (intersecting) Biichi automata. A Biichi automaton accepts some word iff 
there exists an accepting state reachable from the initial state and from itself. 
Hence, we can sum up the model checking problem we consider as the following 
graph-theoretic problem. 

Non-emptiness problem of Biichi automata. 

Given a directed graph G = (V,E), start state (vertex) s & V , a set of accepting 
states F CV, determine whether there is a member of F which is reachable from 
s and belongs to a nontrivial strongly connected component of G. 

The direct approach to solve the problem is to decompose the graph into 
nontrivial strongly connected components (SCCs), which can be done in time 
linear in the size of the graph using the Tarjan’s algorithm nm. However, con- 
structing SCCs is not memory efficient since the states in the SCCs must be 
explicitly stored during the procedure. Courcoubetis et al. |3] have proposed an 
elegant way to avoid the explicit computation of SCCs. The idea is to use a nested 
depth-first search to find accepting states that are reachable from themselves (to 
compute accepting path). The pseudo-code of the NestedDFS algorithm is given 
in Fig. n Only two bits need to be added to each state to separate the states 
stored in VisitedStates during the first and the second (nested) DFS. The ex- 
treme space efficiency of the NestedDFS algorithm is achieved to the detriment 
of time. The time might double when all the states are reachable in both searches 
and there are no accepting cycles. However, in applications to real systems the 
space is actually more critical resource. This makes the nested depth-first search 
the main algorithm used in many verification tools which support the automata 
based approach to model checking of LTL formulas (e.g. SPIN). 

The space requirements of the NestedDFS algorithm are determined by the 
necessity of storing VisitedStates in randomly accessed memory. Several im- 
plementations of NestedDFS use different data structures to represent the set 
VisitedStates. The basic one is a hash table Another implementation H2] 
makes use of symbolic representation of VisitedStates via Ordered Binary Deci- 
sion Diagrams (OBDD). 

Hash compaction is used in |I1> where the possible hash-collisions are not 
re-solved. The algorithm can thus detect a state as visited even if it is not. 
Consequently, not all reachable states are explored during the search, and an 
error might go undetected. 
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proc DFS{s) 

add {s, 0} to VisitedStates; 
foreach successor t of s do 

if {t, 0} not in VisitedStates then DFS{t) fi, 

od ; 

if accepting(s) then seed := s; NDFS{s) fi 

end 



proc NDFS{s) 

add {s, 1} to VisitedStates; 
foreach successor t of s dp 
if {t, 1} not in VisitedStates 
then NDFS{t) 

else if t = seed then “report cycle” fi fl 

od 

end 



Fig. 1. Algorithm NestedDFS 



Another technique which has been investigated to reduce the amount of ran- 
domly accessed memory is state-space caching jS!- The idea is based on the 
observation that when doing a depth-first search of a graph, storing only the 
states that are on the search stack is sufficient to guarantee that the search 
terminates. While this can produce a very substantial saving in the use of ran- 
domly accessed memory, it usually has a disastrous impact on the run time of 
the search. Indeed, each state will be visited as many times as there are simple 
paths reaching it. An improvement on this idea is to store not only the states 
that are on the search stack, but also a bounded number of other states (as many 
as will fit into the chosen “state-space cache”). If the state-space cache is full 
when a new state needs to be stored, random replacement of a state that is not 
currently on the search stack is used. 

The advantage of state-space caching is that the amount of memory that is 
used can be reduced with a limited impact on the time required for the search. 
Indeed, if the cache is large enough to contain the whole state space, there is 
no change in the required time. If the size of the cache is reduced below this 
limit, the time required for the search will only increase gradually. Experimental 
results, however, show that below a threshold that is usually between 1/2 and 1/3 
of the size of the state space, the run time explodes, unless additional techniques 
are used to restrict the number of distinct paths that can reach a given state |n|. 

The behavior of state-space caching is quite the opposite of that of the hash- 
ing technique. Indeed, state-space caching guarantees a correct result, but at the 
cost of a potentially large increase in the time needed for the state-space search. 
On the other hand, hashing never increases the required run time, but can fail to 
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proc DFS(s) 

add {s, 0} to VisitedStates; 
foreach successor t of s do 

if {t, 0} not in VisitedStates then DFS{t) fi 
od ; 

if acccpting(s) then seed := s; NDFS{s) fi; 
if ReductionStrategy{s) then delete {s, from VisitedStates fi 

end 



proc NDFS{s) 

if accepting(s) and {s,0} not in VisitedStates then exit fi; (x) 

add {s, 1} to VisitedStates; 
foreach successor t of s do 
if {t, 1} not in VisitedStates 
then NDFS(t) 

else if f = seed then “report cycle” fi fi 

od ; 

if {s,0} not in VisitedStates then delete {s, 1} from VisitedStates fi 

end 



Fig. 2. Algorithm NestedDFSReSt 



explore the whole state space. A combination of state space caching and hashing 
has been proposed and investigated in [D|. 

In this paper we propose a new technique to attack the state-explosion prob- 
lem using a simple probabilistic method. Actually, the technique has been strongly 
motivated by our intention to improve the performance of the model checker 
SPIN, and the technique has been embedded into SPIN for testing purposes. 

The proposed method allows to solve the emptiness problem of Biichi au- 
tomata (i.e. complete LTL model checking and not only reachability) and it 
never errs. It can be briefly described in the following way. The algorithm is 
based on the nested depth-first search as described in Fig. Ql Each time the 
algorithm backtracks through a state it employs a proper reduction strategy to 
decide whether the state will be kept in the VisitedStates table or whether it will 
be removed. We propose two reduction strategies, the dynamic and the static 
one. While the first one takes on the frequency of visiting the state, the second 
one allows to eliminate delayed storing of the state and thus decreases the num- 
ber of visits of individual states. We specify properties of systems determining 
which strategy suits better for a given verification problem. 



3 Algorithm with Probabilistic Reduction Strategy 

The reason to store the states in the table of visited states during nested depth- 
first search is to speed up the verification by preventing the multiplication of 
work when states are re-visited. A state that is visited only once need not be 
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stored at all, while storing a state which will be visited many times can result in 
a significant speed-up. The standard nested depth-first search algorithm stores 
all visited states. On the other side, the optimal strategy for storing states 
would take into account the number of times a state will be eventually visited 
- a visitation factor. As it is generally impossible to compute this parameter in 
advance, we will use probabilistic method to solve the problem. 

The pseudo-code of the modified nested depth-first-search algorithm with 
reduction strategy, NestedDFSReSt, is given in Fig. El Whenever the DFS pro- 
cedure explores a new state, the state is temporally saved in the VisitedStates 
table (with parameter 0). Whenever DFS backtracks trough a state, a test Re- 
ductionStrategy is performed and if the test evaluates to true the state is removed 
from the VisitedStates table. We will consider two basic probabilistic strategies 
of removing states. The first one dynamieally decides on removing a state each 
time the state is backtracked through, while the second heuristic decides ran- 
domly in advance (before the verification is started) which states will be stored 
permanently. 

As in the case of DFS, the NDFS procedure also needs the list of states 
it has visited to be efficient. Therefore every exploring of a new state results 
in its saving to the VisitedStates table (with parameter 1). Whenever NDFS 
backtracks trough a state it respects the ReduetionStrategy test performed on 
this state by the DFS procedure and if necessary removes the state from the 
table. 

Removing states from the VisitedStates table has direct impact on the time 
complexity of the algorithm as re-visiting a state removed from the table invokes 
a new search from this state. 

The correctness of the NestedDFSReSt algorithm follows from the correctness 
of the NestedDFS algorithm |5]. The additional key arguments it depends on are 
summarized in the following two lemmas. 

Lemma 1. During the whole computation the sequence of states with which the 
DFS procedure is called (DFSstack) forms a path in the graph G. The same is 
true for the NDFS procedure and NDFSstack. 

Proof: The (N)DFS procedure is always called with the argument t which is a 
successor of the current state s. 

Lemma 2. Suppose that during the whole computation both the DFSstack and 
the NDFSstack are subsets of VisitedStates, then the NestedDFSReSt algorithm 
terminates. 

Proof: From the inclusion follows that the (N)DFSstack always forms a simple 
path. The number of simple paths in G is finite and each one is explored at most 
once. 

Theorem 1. The algorithm NestedDFSReSt is eorrect. 

Proof: Whenever the (N)DFS procedure explores a new state, the state is tem- 
porally saved in the VisitedStates table. Therefore (N)DFSstack C VisitedStates 
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is invariantly true and NestedDFSReSt always terminates due to the Lemma 2. 
If NestedDFSReSt reports “cycle” then due to the Lemma 1 there is a reachable 
cycle containing an accepting state. Conversely, suppose there is a reachable cy- 
cle containing an accepting state in G. Deleting states from VisitedStates table 
cannot cause leaving out any call of (N)DFS(t) which would have been performed 
by NestedDFS algorithm. Moreover, the situation in which the condition of the 
if test on the very first line (denoted by x) in NDFS is true is equivalent to 
the situation when {s, 1} is in VisitedStates in NestedDFS algorithm. Therefore 
NestedDFSReSt searches trough all the paths NestedDFS does and thus reports 
“cycle” when NestedDFS does. ■ 

Notice that the test on the first line (x) of NDFS prevents re-searching of 
an accepting state and thus speeds-up significantly the overall time complexity. 
This fact was confirmed also by experimental results. 

The proof of the Theorem [0 is based on the fact that the NestedDFSReSt 
algorithm searches through all the paths the NestedDFS one does. Due to this 
fact our algorithm is compatible with additional techniques used for state space 
reductions, especially with partial order reduction techniques used in SPIN. 

3.1 Dynamic Reduction Strategy 

The pseudo-code implementing the dynamic reduction strategy is as follows: 



funct ReductionStrategy-Dynamic{s) : boolean 
p := random[0^ 1]; 
if P < Pdel 

then ReductionStrategy-Dynamic := true 
else ReductionStrategy-Dynamic := false fl 

end 

Pdel is a fixed parameter determining the probability of deleting a state 
from VisitedStates table. Each time the DFS backtracks through a state s the 
state is deleted with the probability Pdei and is kept stored with the probability 
Psto = 1 — Pdel ■ Once a state is kept stored in the table by the DFS procedure, 
it is never removed. The probability that a state will be eventually stored thus 
depends on the number k of its visits during the computation and is equal to 
Prob{s is eventually stored) = 1 — Pdei- This means that a state with higher 
visitation factor k has also higher probability to be stored permanently. The 
probability that the state s will be re-visited more than i times is equal to 
Prob(s is i times deleted) = P^^i- 

The dynamic reduction strategy would lead to a non-trivial reduction of 
randomly accessed memory if there is a non-trivial subset of the state space 
that will never be permanently stored. The expected memory reduction can be 
expressed as Px {size of the state spaee), where P is the probability that a state 
will never be permanently stored. If k is the average visitation factor then P can 
be estimated as Therefore, we would like to have the highest possible value 
of the probability that a state will never be permanently stored. 



112 



L. Brim, I. Cerna, and M. Necesal 



On the other hand, not saving a frequently visited state increases the time 
complexity of the whole computation. Therefore, we are interested in the ex- 
pected number of visits after which the state is stored permanently. Consider an 
elementary event {s is permanently stored during its i-th visit}. Then 

Prob{{s is permanently stored during its i-th visit}) = P^~i^Psto- 

Let H he & random variable over the above mentioned elementary events defined 
as 

H{{s is permanently stored during its i-th visit}) = i. 

We have that the expected value of H is 

OO OO OO OO 

E{H) = y: ^^del ^sto — Psto ^ 'i'Pdel ~ ^sto 

i—\ i—1 'i'—j 

^ r>j — ^ r> ^ p pO 

_ p ^del _ ^sto pj _ ^sto ^del _ 

Z_/ I _ p I _ p Z_/ del 2 ^ _ p 2 — P^ I 

J = 1 J=0 

Psto Psto 1 

= (1 - Pdeir 

It can be seen that the expected value of the random variable H depends on the 
probability Psto and indicates that value Psto should be high. 

We can conclude that in systems with a high visitation factor we cannot ex- 
pect reasonable space savings without enormous increase of the time complexity. 

3.2 Static Reduction Strategy 

The second strategy tries to eliminate the main disadvantage of the dynamic re- 
duction strategy, namely the delayed storage of a state. If a state will eventually 
be permanently stored, why not to store it immediately during the first visit. 
When deciding which states are to be stored we should prefer states with high 
visitation factor. As we cannot compute this factor in advance we use probabilis- 
tic decision. All states are in advance and randomly divided into two groups: 
states which will be stored and those which will never be stored (represented 
as R). Hence, each state is permanently stored during its first visit or never. 
The ratio between stored and non-stored states is selected with the intention to 
achieve as highest reduction in state space as possible. 

The pseudo-code implementing the static reduction strategy is as follows: 

funct ReductionStrategy-Static(s) : boolean 
it s G R 

then ReductionStrategy-Static := true 
else ReductionStrategy-Static := false fi 

end 

The disadvantage of the static reduction strategy is its insensibility to the 
visitation factor. 
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4 Experiments 

To be able to compare experimentally our probabilistic algorithm with the non- 
probabilistic one, we have embedded the algorithm into SPIN model checker. 

We have performed a series of tests on several types of standard parametrized 
(scalable) verification problems. Here we report on two of them only: 

Peterson. Peterson’s algorithm solves the mutual exclusion problem. We have 
considered the algorithm for parameter N = S determining the number of 
processes. The property to be verified was \3{ncrit < 2) (no more than one 
process is in critical section). 

Philosophers. Dining Philosophers is a model of a problem of sharing of re- 
sources by several processes. We have considered the algorithm for = 4 
and N = 6. The property to be verified was DO^EatingAny = 1) (absence 
of deadlock). 

The other problems we have considered were e.g. the Leader Election problem, 
Mobile processes. In all these experiments we have obtained similar results. 

As our algorithm is compatible with partial order reduction techniques used 
in SPIN we have compiled all problems with partial order reductions. 

For each verification problem we first give two most important characteris- 
tics of the computation performed by SPIN checker: States (the number of states 
saved in the VisitedStates table) and Transitions (the number of performed tran- 
sitions). The number of transition is proportional to the overall time complexity 
of the computation. The size of the VisitedStates table in SPIN’S computation 
is nondecreasing. Once a state is stored in the table it is never removed. On the 
other hand in the NestedDFSReSt algorithm every visited state is temporally 
stored in the table and only when it is backtracked through the (random) deci- 
sion about its permanent storing is made. Therefore for our algorithm we need 
another characteristic, namely the highest size of the Visited States table, Peak 
States. The parameter States declares the number of states stored in the table 
at the end of computation. The remaining two parameters, State Saving and 
Transition Overhead, compare performance of the deterministic algorithm and 
the probabilistic ones. Computations of probabilistic algorithms were repeated 
10 times, presented values are the average ones. 



Peterson’s Algorithm 

Results of experiments are summarized in the Table [D The best results with 
Dynamic Strategy were achieved for storing probability 0.5 where saving in the 
size of stored state space was 33% while increase in the time was negligible, and 
for probability 0.1 with 52% space saving and multiplication factor 4 of time. 
To get deeper inside we mention that the computation without the reduction 
strategy took about 1.5 second in this case. Yet another increase in the deleting 
probability results in substantial grow of time but does not improve space saving 
factor significantly. 
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Table 1. Summary of Experimental Results for Peterson 



States 


Peak 


Saving 


Transitions 


Overhead 


SPIN 


17068 


17068 


0% 


32077 


1.00 


Dynamic Strategy 


Psto = 0.50 


10998 


11421 


33% 


46074 


1.44 


Psto = 0.10 


6724 


8263 


52% 


136344 


4.25 


Psto = 0.01 


5559 


7407 


57% 


1110526 


34.62 


Static Strategy 


Psto = 0.75 


12807 


12812 


25% 


38761 


1.21 


Psto = 0.50 


8568 


9661 


43% 


63662 


1.98 


Psto = 0.40 


6852 


8417 


51% 


390737 


12.18 



Experiments with Static Strategy reveal that we can achieve 43% space saving 
for the price of double time complexity. 51% space saving is attained with worse 
time multiplication factor (12 in comparison to 4) than in the case of Dynamic 
Strategy. The difference between storing probability and real space savings (i.e. 
for storing probability 0.4 we would expect 60% saving instead of measured 51%) 
has two reasons. Firstly, as we do not know which states of the state space are 
actually reachable in the verified system we have to divide the whole state space 
in advance. Secondly, the division determines states which are permanently saved 
but the VisitedStates table contains also temporally saved states and its size can 
be temporally greater (parameter Peak States). State space saving is computed 
via comparing the number of saved states by non-probabilistic computation and 
the peak value of probabilistic computation. 



Dining Philosophers 

Results of experiments are summarized in the Table El for = 4 and in the 
Table i for iV = 6. In both cases the results are comparable. Dynamic Strategy 



Table 2. Summary of Experimeutal Results for Philosophers with = 4 





States 


Peak 


Saving 


Transitions 


Overhead 


SPIN 


3727 


3727 


0% 


18286 


1.00 


Dynamic Strategy 


Psto = 0.50 


3047 


3178 


15% 


33475 


1.83 


Psto = 0.10 


2482 


2686 


28% 


139263 


7.62 


Psto = 0.01 


2316 


2531 


32% 


1287156 


70.39 


Static Strategy 


Psto = 0.75 


2788 


2961 


21% 


49112 


2.69 


Psto = 0.60 


2221 


2577 


31% 


232973 


12.74 


Psto = 0.50 


1875 


2340 


37% 


3285607 


179.68 
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again gives the best results for storing probability between 0.5 and 0.1 . Any fur- 
ther decrease in the storing probability below 0.1 results in significant increase 
of time complexity. In the case of Static Strategy reasonable results were ob- 
tained for storing probability 0.75 and further decreasing of probability leads to 
unreasonable time overhead and thus prevents from higher space savings. 



Table 3. Summary of Experimental Results for Philosophers with N = 6 





States 


Peak 


Saving 


Transitions 


Overhead 


SPIN 


191641 


191641 


0% 


1144950 


1.00 


Dynamic Strategy 


Psto = 0.50 


160426 


165461 


14% 


2152384 


1.88 


Psto = 0.10 


136081 


145214 


24% 


9400300 


8.21 


Psto = 0.01 


131306 


140758 


27% 


91533400 


79.90 


Static Strategy 


Psto = 0.75 


143661 


155920 


19% 


6702840 


5.85 


Psto = 0.65 


124377 


143691 


25% 


116103466 


101.40 



Generally, the results for Philosophers are worse than those for Peterson’s 
algorithm and are remarkably influenced by the visitation factor. While in the 
Peterson’s algorithm the average number of state visits in SPIN’S computation 
is 32077/17068 = 1.8, in Philosophers it is 4.9 {N = 4) and 6 {N = 6). Experi- 
mental observations are thus in accordance with deduced theoretical results. 



5 Random Nested DFS 

Besides the algorithm with probabilistic reduction strategy we have also explored 
the potential of randomizing the branching points in nested depth first search. 
Verification tools typically build the state space from the syntactical description 
of the problem. E.g. in SPIN the foreach successor t of s do in the depth first 
search is implemented as for i = Iton cycle. This means that the search order is 
fixed by the input PRO MELA program describing the system. If the verification 
fails due to space limitations it is recommended to re-write the program to 
re-order the guarded commands in conditionals and loops. However, the user 
typically has no information on what would be a good re-arrangement. Hence, 
the situation is very suitable for a randomized approach. 

We have implemented the foreach successor t of s do in the depth first 
search as a random selection of the successors ordering and performed a series of 
comparisons with the standard SPIN tool on similar set of problems as we did 
before. Even though the method is trivial, the results we obtained were quite 
surprising. For instance for the Philosophers (with an error) the results are 
partially summarized in the Table 0 
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Table 4. Summary of Experimental Results for Random Nested DFS 





SPIN 


Random NDFS 


N 


States 


Trans 


Memory 


Runs 


Success 


States 


Trans 


Memory 


11 


288922 


1449200 


56.9 MB 


10 


10 


100421 


505150 


26.4 


12 






205.0 MB 


10 


3 


68355 


346824 


19.9 


14 






2.8 GB 


50 


5 


46128 


250266 


16.2 


16 






38.5 GB 


50 


5 


46288 


245406 


17.8 


20 






6.7 TB 


50 


2 


38282 


213639 


18.2 



For the value of the parameter N greater than 11 the SPIN model checker 
was not able to complete the computation. We therefore give estimated values for 
the memory requirements obtained by extrapolation from finished computations. 
The randomized algorithm was repeatedly performed {Runs) and the number 
of successful runs (discovering the error before memory overflow) is reported 
{Success). The experiments indicate that even a small number of repetitions can 
dramatically increase the power of the tool. 

We have also considered some artificial verification examples, which demon- 
strate the potential of the method in some extreme cases. Consider the following 
verification problem defined by the program 

1 proc ExIF 

2 MainCounter := 0; StepCounter := 0; 

3 while StepCounter < 1000 do 

4 if 

5 true — )> MainCounter := MainCounter -|- 1 

6 true — )> MainCounter := MainCounter -|- 2 

7 fi; 

8 StepCounter := StepCounter + 1 od 

9 end 

and the LTL formula 



D{MainCounter < 2000 — Diff) 



The parameter Diff determines the ratio of runs of the program that satisfy 
and violate the formula. More precisely, the probability that MainCounter = 
2000 - Diff is 



Prob{MainCounter = 2000 — Diff) 





1000 



We have performed experiments for various values of Diff. The results are sum- 
marized in the Table 0 The experiments have confirmed that the actual memory 
savings strictly depend on the value of the parameter Diff, that is on the proba- 
bility of a faulty run, and have ranged from 20% up to 90%. We stress that after 
re-ordering of guarded commands in the ExIF program (swapping lines 0 and EJ 
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SPIN finds the counterexample immediately. Re-writing the program helps in 
this case. The next example shows that in some situations even re-writing the 
program does not help. 



Table 5. Summary of Experimental Resnlts for ExIF 



Diff 


ViolProb 


Algorithm 


States 


Transitions 


% 


400 


1.3642.10"^° 


RandNestedDFS 

SPIN 


37999 

363202 


55810 

543502 


10.4% 

100.0% 


200 


8.2249.10-®® 


RandNestedDFS 

SPIN 


368766 

643602 


551537 

964002 


57.3% 

100.0% 


100 


6.7017.10”^®^ 


RandNestedDFS 

SPIN 


647855 

813802 


969977 

1219250 


79.6% 

100.0% 



Let us consider the LTL formula 

\3{{StepCounter < 1000) V {MainCounter yf 1500)). 

The formula expresses the property that at the end of every computation (i.e. 
when StepCounter = 1000) the value of MainCounter is not 1500. It is easy to 
see that the ExIF program does not fulfil this property. The erroneous computa- 
tions are those where both guards are selected equally. For every re-ordering of 
the guards SPIN has to search the significant part of the state space to discover 
a counterexample. On the other hand, the RandNestedDFS algorithm successes 
very quickly as it selects both guards with the same probability. The same effect 
has been observed in other tests as well. E.g. in the Leader election problem, for 
every permutation of all guards SPIN has searched approximately the same num- 
ber of states while RandNestedDFS has needed to search through significantly 
smaller part of the state space. 

6 Conclusions 

While verification of probabilistic systems seems to be ready to move to the 
industrial practice, the use of probabilistic methods in model checking of non- 
probabilistic systems is at its beginning. The use of probabilistic methods in the 
explicit state enumeration techniques to reduce the memory required by hashing 
is an excellent example of the potential of probabilistic methods. Our intention 
was to investigate other possibilities how randomization could help in model 
checking. 

We have proposed a new probabilistic verification method which could reduce 
the amount of random access memory necessary to store the information about 
the system. The reduction rate depends on the verified system, namely on the 
average number of state visits. Our experiments have confirmed that the method 
could achieve a non-trivial reduction within reasonable time overhead. 
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Another important issue for further study is to examine possibilities of com- 
bining our probabilistic reduction strategy algorithm with other techniques to 
reduce memory usage. We also plan to perform additional experiments to give a 
more comprehensive view of the performance of our technique and of its scala- 
bility. 
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Abstract. In this paper we present a representation of the Markov 
process underlying a PEPA model in terms of a Kronecker product of 
terms. Whilst this representation is similar to previous representations 
of Stochastic Automata Networks and Stochastic Petri Nets, it has novel 
features, arising from the definition of the PEPA models. In particular, 
capturing the correct timing behaviour of cooperating PEPA activities 
relies on functional dependencies. 



1 Introduction 

Performance investigation of modern computer and communication systems re- 
quires the development of relevant and efficient modelling techniques. The rich 
synchronisation constraints and the size of these systems lead to complex mod- 
els with exponential growth of the number of states. Traditional performance 
models, based on queueing networks, cannot readily capture these constraints; 
thus several new performance modelling techniques have been developed, e.g. 
Stochastic Petri Nets (SPN), Stochastic Automata Networks (SAN) and Stochas- 
tic Process Algebras (SPA). 

Petri nets were designed to represent synchronisation constraints within con- 
current systems and protocols; SPN associate random variables with timed tran- 
sitions within the net mm - However, although the graphical representation 
of Petri Nets presents the dynamic behaviour of the model, it provides little 
insight into the structure of the system being modelled. 

SAN and SPA provide mechanisms which allow the increasing complexity of 
synchronisation constraints to be captured whilst retaining the compositional 
structure of the system explicitly within the model. For many modern systems, 
being able to construct a model from components or elements, reflecting the 
system’s composition, greatly aids handling the complexity of the model con- 
struction task. As with all state-based modelling formalisms, such models are 
prone to state space explosion. However, both formalisms incorporate techniques 
for overcoming this problem. 

* This work is supported by CNRS/RS project (UIIVV 78171) and EPSRC project 
COMPA (G/L10215). 
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The SAN formalism, developed by Plateau CHI, models complex systems 
with interacting components such as parallel systems. To tackle the state space 
explosion problem, Plateau nn has proved that the generator matrix of the 
Markov process underlying a SAN model can be analytically represented us- 
ing Kronecker algebra. Moreover the solution of the model can be achieved via 
this tensor expression of submatrices — the complete matrix does not need to be 
generated. 

SPA are extensions of classical process algebras such as CCS and CSP, analo- 
gous to SPN in the sense that random variables are associated with timed actions 
in the model. In this paper we consider the Markovian process algebra. Perfor- 
mance Evaluation Process Algebra (PEPA), introduced by Hillston in 1994 [11) . 
In PEPA, a system is described as an interaction of components which, either 
singly or multiply, engage in activities. The components represent the active 
parts within the system and the activities the actions of those parts. 

Various techniques for solving large models have been developed for PEPA 
but these have focused on aggregation or decomposition techniques, which use 
the process algebra structure of the model to guide manipulations of the un- 
derlying Markov process. In this paper we show that a PEPA model can also 
be represented analytically using Kronecker algebra and solved without con- 
structing the complete generator matrix. Correct representation of the features 
of the PEPA model, in particular the synchronisation behaviour, relies on the 
functional dependencies introduced in PEPA formalism in HS|. Just as for SAN 
models, we show that the translation from the model to the compact represen- 
tation is automatic. 

This paper is structured as follows. In Sect.El we present the PEPA language. 
A small example illustrates the use of this modelling technique. Section O is 
dedicated to the functional depencies in PEPA. In Sect. E] we show how to 
represent the underlying Markov process of a PEPA model using the tensor 
algebra. An application example is given, followed by the proof of the validity 
of this analytical representation. Section 0 is dedicated to related work. Finally, 
we conclude with some remarks and future work. 



2 PEPA 

The basic elements of PEPA fH) are components and activities, corresponding 
to states and transitions in the underlying Markov process. Each activity has 
an action type and r denotes the distinguished type representing private or 
unseen action. The duration of each activity is represented by the parameter 
of the associated exponential distribution: the activity rate of the activity. The 
rate may be any positive real number, or the distinguished symbol T (read as 
unspecified). Thus each activity, a, is a pair (a,r) where a is the type and r is 
the rate. We let C denote the countable set of components and A denote the 
countable set of all possible action types. We denote by Act C Ax K.+ , the set 
of activities, where R+ is the set of positive real numbers plus the symbol T. 
Models in PEPA are built using a small but expressive set of combinators: 



122 



J. Hillston and L. Kloul 



Prefix (a, r).Ci. Prefix is the basic mechanism by which the behaviours of com- 
ponents are constructed. The component carries out activity (a,r) and subse- 
quently behaves as component Ci. 

Choice Ci + C2- The component represents a system which may behave either as 
component Ci or as C2: all the current activities of both components are enabled. 
A race condition determines the first activity to complete, and so distinguishes 
one component; the other is discarded. 

Cooperation C\ C2. The components proceed independently with any activi- 
ties whose types do not occur in the cooperation set L. However, activities with 
action types in the set L require the simultaneous involvement of both compo- 
nents; these shared activities are only enabled when they are enabled in both 
Cl and C2. The shared activity occurs at the rate of the slowest participant. 
The capacity of a component Ci to carry out a given action type a (the sum of 
rates associated with its a actions) is called its apparent rate, denoted ra{Ci). 
The apparent rate of a shared activity is the minimum, among the participating 
components, of the apparent rates for that type. 

If an activity has rate T the component is passive with respect to that action 
type and it does not influence the rate at which such shared activities occur. 
When the set L is empty, we use the more concise notation C\ || C2 to represent 
Cl ^C 2 . 



Hiding CijL. The component behaves as Ci except that any activities of types 
within the set L are hidden, i.e. such an activity exhibits the unknown type r 
and the activity can be regarded as an internal delay by the component. The 
original action type of a hidden activity is no longer accessible; the duration is 
unaffected. 

Constant M = Ci. Constants are components whose meaning is given by a 
defining equation: M = Ci gives the constant M the behaviour of the compo- 
nent Cl. 

The semantics of PEPA, presented in the structured operational semantics 
style, are given in m The underlying transition system also characterises the 
Markov process represented by the model. Rules are given for each of the com- 
binators, showing how the component may evolve. Here we show only the rule 
for shared activities (Fig. Q)- 



p CXI g P' ex Q’ 



(a € L), 



r = 



ri V2 

ra(P) rdQ) 



mm{ra{P),ra{Q)) 



Fig. 1. Operational rule defining shared activities 



An Efficient Kronecker Representation for PEPA Models 123 

From a model definition M we can apply the semantic rules exhaustively 
to find the complete set of reachable states, the derivative set of M, ds{M). 
From this set, we can construct the derivation graph. The derivation graph is a 
directed multigraph whose set of nodes is ds{M) and whose arcs represent the 
possible transitions between them. To derive a Markov process from a PEPA 
model we associate a state with each node of the derivation graph. Action type 
information is discarded so that edges are labelled only by rates; multiple edges 
between a pair of nodes are combined by summing the corresponding rates. The 
rate on an edge in this modified graph becomes the corresponding entry in the 
infinitesimal generator matrix. Thus the rate between components C and C is 
denoted q{C,C). Similarly the conditional transition rate between C and C' 
due to activities of type a is denoted q{C, C , a). 

Necessary (but not sufficient) conditions for the ergodicity of the Markov 
process in terms of the structure of the PEPA model have been identified and 
can be readily checked P). These imply that the model should be constructed 
as a cooperation of sequential components, i.e. components constructed using 
only prefix, choice and constants. Thus the compositional structure of PEPA 
models is at the level of the cooperating components; we refer to such models 
as well-defined. Syntactic analysis can be used to determine all the action types 
which will occur within the lifetime of a component C, a set denoted •A{C). 

Example. Consider a simple two-place buffer and a server. The buffer accepts 
arrivals with a rate A and passes the contents for service. When there are two 
customers in the buffer each attempts service, but only the front customer can 
be successfully serviced with rate s. Service of the second customer results in 
a partial service which must be corrected, at rate t before the buffer can make 
any other action. The server simply accepts customers for service (passively) and 
then allows them to depart, carrying out a false departure after a partial service. 
Let d be the departure rate. 

In PEPA, the system is represented as the interaction of two components 
BufferQ and Server. We use three action types: in, service and depart. The first 
describes the arrival of a new customer in the buffer, the second, the service 
completion, and the last one the departure of a customer from the server. The 
components are defined as shown below. 

BufferQ = {in,X).Bufferi Server = {service, T). Server 

Buffer^ = {service, s).BufferQ + {in, X).Bujfer2 Server = {depart, d). Server 

Buffer2 = {service, s).Bufferi + {service, s). Buffer^ 

Buffer^ = {service, t) .Buffer I 

In addition to the mutually recursive sets of equations defining the behaviour 
of each sequential component, we have a system equation which defines the 
cooperation between the two components. 



System = Buffer^ Server 

{service} 
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3 Functional Dependencies 

In SAN, automata are able to influence one another in two ways, both related to 
events. Direct interaction between automata is modelled by synchronised tran- 
sitions, equivalent to cooperation or shared activities in PEPA. 

The other form of interaction is less direct: transition rates within an au- 
tomaton can be influenced by the local states of one or more other automata 
of the network. Using such rates may lead to a reduction in the model size 
since functional rates are a means to avoid explicitly modelling all parts of a 
system’s bahaviour. This benefit is most appreciated when building/solving the 
underlying Markov chain. In HS|, we have introduced the notion of functional 
dependencies between PEPA model’s components by extending the activity rates 
to include functional rates. 

In PEPA the set of activities Act is defined as Act C A x M+ where is 
the set of positive real numbers defined as follows: 

K'*' = {r|r > 0; r G M} U {T} 

In the context of PEPA, a functional dependency may involve one or several 
components. In a functional dependency involving a single component, the rate 
value of one or several activities of the component depends on the current state 
of the component itself. This captures the presence of several apparent rates 
for an activity in a component. In this type of functional dependency, the rate 
value expressed as a function of the current component state is still a positive 
real number and can never be zero. However, this may not always be the case 
if the functional dependency involves two or more components. For example, a 
functional dependency between two components means that the behaviour of 
one component depends on the current state of the other one. This implies that 
either the activity to be performed by the first component and/or its rate value 
will be determined by the current state of the second component. The rate value 
may then be any non-negative real number of R'*" including zero, particularly 
when the choice of the activity to be performed is done according to the state 
of another component. 

The introduction of functional dependencies in PEPA requires us to relax 
the constraint on the definition domain of an activity rate [IS|. Thus, the set of 
activities Act is now defined as Act C A x R* where R* is the set of positive 
real numbers defined as follows: 

R* = {r\r > 0;r G R} U {T} 

For more details about the impact of functional dependencies on PEPA mod- 
els and the aggregation technique see m 

Example. Consider again the system of the previous example. If we opt for 
functional rates, the buffer may then be modelled differently: 
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Buffer^ = {in, X). Buffer^ 

Buffer-y = {service, f x p).BufferQ + {in, X).Bujfer 2 
Bujfer 2 = {service, f x p).Bufferi + {service, f x p).Bujfer^ 
Buffer^ = {service, f x p).Bufferi 



where / is a function of the state i, i = 0, . . . , 3 of component Buffer such that: 



Note that the definition of the Server, and the equation defining the complete 
system behaviour, remain unchanged: 



This example shows that the introduction of functional rates in PEPA models 
allows us to avoid having different apparent rates for an activity within a single 
component. Whereas in the first version of the model, activity service has two 
apparent rates (s and t), the same activity has only one apparent rate (/) when 
using functional rates. 

The association of an apparent rate with each action type within a single 
component leads, as we will show in Sect.0 to a simplified tensor representation 
of the generator matrix associated with a PEPA model. 

4 Tensor Representation 

In this section we establish how to represent the infinitesimal generator matrix 
corresponding to a PEPA model as a sum of tensor products, analogous to the 
representation of a SAN. We proceed in three steps. In the first, we consider only 
the non-shared activities which do not belong to any cooperation set, represent- 
ing the independent aspect of a component’s behaviour. For each component we 
capture its local transitions in an appropriate matrix. In the second step, we 
consider the activities which belong to at least one cooperation set. This will 
allow us to take into account, in our tensorial representation, the interactions 
between the components. We represent each type of interaction by the tensor 
product of matrices capturing each component’s capacity to participate in a 
shared activity. Using the obtained results, we finally show how to represent the 
global generator matrix of a complete model. 




and p is a probability function defined as: 




System = Buffer^ IXI Server 

^ U f^scrvice} 
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4.1 Non-interacting PEPA Components 

We define a non-interacting component as a component for which at least one of 
its activities is a non-shared activity, or for which all its activities are cooperating 
activities, but there exists at least one non-shared activity in the model in which 
this component does not participate. 

With each non-interacting component Q, i G { 1 , A}, we associate a gen- 
erator matrix Ri of size ni x with ni = |ds(Ci)|. If this component has at 
least one non-shared activity, the elements of its matrix are the rates of its indi- 
vidual activities. Otherwise, the matrix associated with this component is a null 
matrix. In both cases, the resulting matrix describes the local transitions of the 
component. 

Now consider a PEPA model M = C\ || C2 || ... || Cn and assume that Ci, 
C2, ■ . ., Cn are represented by infinitesimal generator matrices R\, R2, . . ., Rn 
respectively. Then any state of M can be represented as (Ci , C2J2 > • ■ • > ^N,jN) 
where ji G { 1 , 2 ,..., rii} for 1 < i < N. Moreover, the system of the N non- 
interacting components may be characterised by the infinitesimal generator ma- 
trix jlZj 

N N 

Q — Rk — ^ ^ 0***0 Ink — \ O Rk O Ink^\ 0***0 

k—1 k—1 

N k-1 N 

k—1 i—1 i—k-\-l 

where Id is the identity matrix of size d. © and O are the tensorial sum and 
product operators respectively El. 



4.2 Interacting PEPA Components 

Most useful PEPA models are comprised of components which interact. To rep- 
resent the interacting part of the component’s behaviour, we associate with each 
action type a in O, the set of cooperating action types, a transition probability 
matrix Pi^a- This matrix captures the capacity of component Ci to participate 
in the shared activity a. Thus, each element of this matrix represents the transi- 
tion probability of component Ci with activity a with rate raiCi). Note that if 
a component does not participate in a shared activity a, the matrix associated 
is an identity matrix. 

In general, if a PEPA model is composed of N components, the interaction 
between these components can be expressed as follows: 

N 

Ta Pi, a 

ol^Z i—1 
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where Tq is the minimum of the functional rates of action type a over all com- 
ponents Ci, i = 1 . . . N: 



4.3 Global Generator Matrix Representation 

Now consider a PEPA model composed of interacting and non-interacting com- 
ponents. The corresponding generator matrix may be represented using Kro- 
necker algebra as stated in Definition ^ 

Definition 1. The generator matrix Q of the Markov chain associated with a 
PEPA model is 



— N is the total number of components in the PEPA model and Z the set of 
cooperating action types, determined syntactically. 

— Va is the minimum of the functional rates of action type a over all compo- 
nents Ci, i = 1 . . . N . 

— Ri is the transition matrix of component Ci relating solely to its individual 
activities. 

— Pi^a is the probability transition matrix of component Ci due to activity of 
type a. Its elements’ values are between 0 and 1. 

— Pi^a is a matrix representing the normalization associated with the shared 
activity a in component Ci . 

Unlike the local transition matrices Ri, the cooperation matrices Pi^a sxe 
not generators. So we need to introduce diagonal corrector matrices Pi^a to 
normalize the cooperation matrices, i.e. ensure that row sums are zero. This is 
shown in (Unj. 

In order to apply the equation above we must place a restriction on the use of 
types within cooperation sets, to ensure that each action type uniquely defines 
a synchronisation event. To see the need for this restriction, consider the model 
M = [V S) II (T ^ U). Assuming that each component enables a with just 
one apparent rate ra{V), Ta{S), etc., applying the equation above, we write the 
generator matrix of this model as follows: 



= min{ra{Ci),rc{C2), . . .,ra{CN)) 




(4.1) 



where 



4 




— Pv,a ® Ps,a ® Pr.a ® Pu,a) 
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It is clear that in this representation all the components are forced to make 
the same a cooperation. However, applying the semantics, there are two poten- 
tial shared a activities: one involving V and S and one involving T and U . These 
can proceed concurrently with each other. 

Thus we preprocess the model: when the same action type appears in distinct 
cooperation sets we rename the action type in the appropriate components and 
cooperation sets so that they are distinguished in Z. For example, in the model 
above, we might distinguish (affecting V and S) and (affecting T and U) 
and rename all a activities in V, S, T and U appropriately. 



4.4 Example 

Consider the two place buffer and the server described in Sect. El In the following 
we show how we construct the tensor expression for the global generator matrix 
of the corresponding model. 

The model has two components, each component has two action types in its 
complete action type set: in and service for Buffer and service and depart for 
Server. The type service is the only element of Z, the set of cooperating action 
types; the other action types being local to their respective components. Firstly 
we construct the matrices representing these local activities as follows: 



^Buffer 



/-A A00\ 
0 -A A 0 
0 0 0 0 
\ 0 0 0 0 / 



R 



Server 




0 

d 



When we come to represent the cooperations we consider each action type 
in the cooperating set. In our case, this set is composed of only one action 
type a = service. Component Buffer participates to this activity with rate 
rffBuffer) = / whereas component Server participates to this activity with 
rate rffServer) = T. According to the semantic of PEPA, the resulting rate 
Tq, of the shared activity is r^ = min{f, T) = /. Thus the Buffer component’s 
contribution and the Server component’s contribution to the cooperation are 
expressed by respectively: 



P 



Buffer, a 



/O 0 0 0\ 
10 0 0 
O^Oi 



and 



P 



Server, c 



0 1 
0 0 



\0 1 0 0 / 

The corresponding normalising matrix pairs are straightforward to construct: 



P 



Buffer,a 



/O 0 0 0\ 
0 10 0 
0 0 10 
\0 0 0 1 / 



and 



P 



Server,a 



1 0 
0 0 
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Thus the complete expression becomes 

/-A A 0 0\ 

0 -AAO /O 0 
0 0 0 0 ®\d-d 
V 0 0 0 0/ 



Q = 



+ /X 



/O 0 0 0\ 
10 0 0 
o^oi 



\0 1 0 0 / 

leading to the complete generator matrix: 

/-A 0 A 

d — (d+A) 0 

0 s — (s + A) 



/o lA 


/O 0 0 0\ 
0 10 0 


(IQ\ 


(ooj 


0 0 10 


^[oo) 




1^0 0 0 1^ 





Q = 



0 

A 

0 



0 

0 

A 



0 
0 
0 
0 
V 0 



-(d + A) 0 
s 
0 
t 
0 



0 0 0 \ 
0 0 0 



0 0 0 
A 0 0 
-2s 0 0 s 

d -d 0 0 

0 0 -t 0 

0 d — d / 



0 



Let us consider a modification of the model in which the Server, instead of 
being passive with respect to service has a local rate x, such that s < x <t. Then 
the construction of the tensor expression proceeds in exactly the same way except 
that when we come to compute the resulting rate we obtain is r^ = min{f,x) 
which value depends on the current state of component Buffer. According to 
this, the generator matrix is: 



Q = 



/-A 0 A 

d — (d+A) 0 

0 s —(s + A) 

0 0 d 

0 0 0 

0 0 0 

0 0 0 

\ 0 0 0 



0 0 0 0 0 \ 

A 0 0 0 0 

0 A 0 0 0 

-(d+A) 0 A 0 0 

1 -X 0 0 I 

0 d -d 0 0 

X 00 —X 0 

0 0 0 d -d/ 



4.5 Validity of the Kronecker Expression 

In this section we prove the validity of the tensor expression Q given in 
i.e. we show that for all reachable states the tensor expression gives us the 
same transition rates as the generator matrix Q* derived from the semantics of 
PEPA via the labelled transition system. First, we establish some notation and 
terminology. 

A PEPA model is given by components (C'i)ig[i..Ar], each with state space 
Si = dsiCi) and a model equation M = Ci IXI • • • IXI Cm- Syntactic analysis 

L-i 
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readily identifies Z, the set of cooperating actions. For each component Ci we 
can identify the cooperations it must participate in, a set we denote Zi, where 
Zi = Z (1 A{Ci). Conversely, for each cooperating action type a, we denote by 
Z(a) the set of components which participate in a typed activities. 

For the same model we may have two views of the global state space. The 
first is ds{M), the derivative set of M generated by the operational semantics 
via the labelled transition system. The second is S' = the product state 

space, generated directly from the derivative sets of the components. In general, 
the constraints placed on the model by cooperation will mean that ds{M) C S, 
i.e. S will contain unreachable states. 

We suppose that every local state space Si is ordered — simply take the order 
generated by the breadth-first search carried out in the PEPA Workbench to 
build the labelled transition system. In the following we assume that both ds{M) 
and S are ordered lexicographically according to the ordering within component 
state spaces and the vector representation of the state space. 

We will write C to denote a vector (Ci, . . . , C^), and C[Ci := C-] to denote 
the vector obtained from C by substituting C[ for Ci. We denote by Q*, the 
original transition matrix of M defined as follows: 

— For all C, C' G ds{M) such that C yf C', Q*(C, C') is the transition rate as 
usually defined for PEPA, the sum of activity rates on arcs linking C and 
C' in the derivation graph: 



Q*(C,C')= ^ r 



The set of all transitions(activities) can be partitioned into individual and 
shared transitions, shared transitions can be further partitioned by action 
type. Thus the off-diagonal elements of Q* can be expressed as a sum of 
matrices as follows: 



N 

g‘(c, c') = g*(c, c') + Ql{c, C) 

i=l aG2 



where: 



g*(C,C')= ^ r 

c(fv)c/ 

Ql{C,C') = q{C,C',a) 



where = C[Ci := Ci'] and a ^ Z 



where a £ Z 



— For all C G ds{M), Q*{C, C) is calculated such that the sum of the elements 
in a row of Q* will be zero. 



Theorem 1. Consider a well-defined PEPA model with N components inter- 
acting via a set of action types Z as defined by a model equation 
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Then the original transition matrix Q* is such that, for all reachable states, 
C, C' G ds{M), Q*{C, C') = Q(C, C') where Q is defined as 

N / N N 

Q ® + X! ’min{ra{Ci), . . .,ra{CN)) ( Pi,a 

i—1 OL^Z, \i— 1 i—1 

Furthermore, for C G ds{M) and C' ^ ds{M), Q{C,C') = 0, i.e. there are 
no transitions from reachable states to unreachable ones represented in the tensor 
expression. 

Proof. We can re-express Q as follows: 

N 

g = ^G,+ ^G„-^ G„,„ 

i—1 ol^Z 



where 



i-l 


N 




<S) 

II 




(4.2) 




j=i+l 




N 






Get — a ^ (^^Pi,ot 
i—1 




(4.3) 


N 






a,n — '^a ^ 




(4.4) 


Z=1 







and ra = mm(r„(Gi), . . .,ra{CN)). 

First, we consider the non-diagonal elements of the matrices. We will find it 
convenient to use kronecker functions: 



5{x,y) 



1 \i X = y 
0 \i X ^ y 



Individual transitions. From above, 



i-l N 

G* = (g) 0 (g) In 

j=l j=i+l 



G,(C, C') = i?,(G„ G/) X l[ 6{Ck,Ck') 

k = l 

k^i 

By the definitions of Ri and Q*, it follows that, for all C G ds{M), for all 
*G{1,...,1V}, 



G,(C,C') 



Q*(Gi,G/) if C' = C[Gi := G/] 
0 otherwise 
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Clearly, C' = C[Cj, Ci] implies that C' is reachable. Thus it follows that for 
C, C' G ds{M), for local transitions, i.e. those involving only one component Ci, 
the off-diagonal elements of Q* and Gi are identical. Moreover, for C G ds{M) 
and C' i ds{M), G*(C,C') = 0. 



Cooperating transitions. From above, 

N 

X (g) (4.5) 

i^l 

Thus 

N 

G„(C,C') =r„ xl[Pi^^{C,,Cl) 

k=l 

Since a component i does not participate in activities of type a if t ^ 2i{a), we 
can rewrite this as: 

G„(C,C')=r„x n PiACi,C')x n S{Ci,C') 



Recall that Q* (C, C') = q{C, C', a), where a € Z. If we consider the seman- 
tic rule governing cooperation we can see that this transition rate consists of the 
minimal apparent rate of the participating components multiplied by the con- 
ditional probability that C' is the derivative resulting from the transition. This 
conditional probability is the product of the conditional probabilities in each 
component, since we assume that each component chooses between instances of 
a independently. For component Ci, this conditional probability is expressed as 



p(Ci,C',a) 



g(Ci,C',a) 

q(Ci,a) 

1 

0 



if i G Z{a) 

if i ^ Z(a) and Ci = C' 
otherwise 



Thus, for all C G ds{M) 
Ga(C,C') 



X p{C,C',a) if C' G ds{M) 
0 otherwise 



and so, for C, C' G ds{M), 

G„(C,C') =r„ xp(C,C',a) 



It follows that for all C G ds{M), 



G„(C,C') 



Q* (C, C') if C' G ds{M) and C C' 
0 otherwise 



In particular, for C G ds{M) and C' ^ ds{M), Gq(C,C') = 0 so there are no 
cooperating transitions into unreachable states from reachable ones. 
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Diagonal Elements. Finally, we consider only reachable states C G ds{M) and 
show that Q*(C, C) = Q(C, C). However, this follows immediately since by the 
previous arguments the off-diagonal elements of rows corresponding to C in Q* 
and Q are in one-to-one correspondence, and furthermore, in each matrix the 
diagonal elements are chosen to normalise the matrix. For Q, Gi is already a 
generator whereas we have introduced Ga,n to normalise Ga- Thus by construc- 
tion it follows that for all reachable states C G ds(M), Q*(C, C) = Q(C, C), as 
required. 

□ 

According to the tensorial form of the generator, we store at most E entries: 

N 

E={1 + 2\Z\)J2s! 

i=l 



4.6 Solution Techniques 

The tensorial representation of the generator matrix corresponding to a SAN 
model was proposed in 1984 by Plateau M- Since then different solution tech- 
niques have been investigated and several of them have been adapted to the 
context of this compact representation. 

The main solution techniques used are either iterative methods such as the 
power method and Gauss Seidel or projective methods such as the Arnoldi and 
the GMRES methods. 

In prj, the problem of computation time has been addressed. It has been 
shown how the methods of Arnoldi and GMRES can be used to substantially 
reduce the number of iterations needed when compared with the power method. 
Moreover, several preconditioning strategies that may be used to speed the iter- 
ation process even further have been investigated. 

The power method, Arnoldi and GMRES methods have been incorporated 
in the tool PEPS implemented by Plateau’s team. For each method, versions 
both with and without matrix preconditionning have been implemented. 

The tensorial representation we propose for PEPA models is very similar 
to the one developed by Plateau for the SAN formalism. Therefore the solution 
techniques adapted to the tensorial representation of SAN and the computational 
results such as those presented in m about the impact of the functionnal rates 
on the Descriptor-vector multiplications in SAN may be, without doubt, applied 
in the context of PEPA formalism. 

5 Related Work 

Kronecker algebra representations have been used for some time as a means 
to address the state space explosion problem arising in the numerical solution 
of Markov chains. As mentioned earlier, the pioneering work in this area was 
carried out by Plateau on Stochastic Automata Networks ^H|- More recently. 
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Kronecker-based solution techniques have been developed for various Petri net 
based-formalisms, for example [I8I4I5I6| . 

With their explicit compositional structure, SPAs would appear to be natural 
candidates for Kronecker representation; however, there is little previous work 
on this topic. In 1994 Buchholz proposed an SPA called MPA, for which the 
mapping to an underlying Markov process is only defined in terms of a tensor 
expression 0. However, in MPA the interpretation of both basic actions and 
shared actions is quite different to that in PEPA, chosen specifically to facilitate 
the tensor representation and without a natural modelling interpretation. MPA 
has not been developed further. In this approach the usual labelled transition 
system semantics is avoided and so there was no need to show the validity of 
the tensor expression with respect to the standard Markov process generation 
procedure. A similar denotational approach to semantics, making use of tensor 
expressions, is developed in the work of Rettelbach and Siegle uni- 

In |0| El-Rayes presents an extension of PEPA and an associated solu- 
tion technique based on the Matrix-Geometric Method (MGM). Her language 
PEPA^ allows exponential durations to be replaced by phase type distributions. 
In her mapping to the underlying Markov process, these distributions are repre- 
sented by Kronecker expressions within the block-structured matrices used for 
the MGM. This is distinct from the use of Kronecker expressions in this paper. 



6 Conclusions 

In this paper we have presented a mapping from a SPA formalism to a Kronecker 
representation. Ours is the first such mapping aimed at implementation and 
incorporation into a tool. The SPA we use is PEPA and the mapping is specific 
to that formalism due to the complex semantic rules defining synchronisation 
between PEPA components. Whilst other SPAs such as EMPA 0 and IMG |T^ 
have apparently simpler rules of synchronisation, they include immediate actions 
which would complicate the mapping to a Kronecker representation. 

Once the prototype implementation of this approach is incorporated into the 
PEPA Workbench 0, we aim to improve its efficiency. In particular we plan 
to investigate the multilevel approaches which have been employed with SPN 
to avoid the incorporation of unreachable states. We will also be interested in 
investigating techniques which exploit this Kronecker representation to solve the 
model efficiently and in comparing our approach with other compact represen- 
tations, such as those based on HDDs 113 - 
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Abstract. In this paper we extend a performance measure sensitive 
Markovian bisimulation congruence based on yield and bonus rewards 
that has been previously defined in the literature, in order to aggregate 
more states and transitions while preserving compositionality and the 
values of the performance measures. The extension is twofold. First, we 
show how to define a performance measure sensitive Markovian bisimu- 
lation congruence that aggregates bonus rewards besides yield rewards. 
This is achieved by taking into account in the aggregation process the 
conditional execution probabilities of the transitions to which the bonus 
rewards are attached. Second, we show how to define a performance 
measure sensitive Markovian bisimulation congruence that allows yield 
rewards and bonus rewards to be used interchangeably up to suitable 
correcting factors, aiming at the introduction of a normal form for re- 
wards. We demonstrate that this is possible in the continuous time case, 
while it is not possible in the discrete time case because compositionality 
is lost. 



1 Introduction 

In the past five years the problem of specifying performance measures has been 
addressed in the field of Markovian concurrent systems. Following PH], in |S1 
m the performance measures are characterized through atomic rewards to be 
suitably attached to the states and the transitions of the Markov chains (MCs 
for short) associated with the Markovian process algebraic specifications of the 
systems. While in m the states to which certain rewards have to be attached 
are singled out by means of temporal logic formulas to be model checked, in |3] 
rewards are directly specified within the actions occurring in the specifications 
and are then trasferred to the MCs during their construction. In [El, instead, 
temporal reward formulas are introduced, which are able to express accumu- 
lated atomic rewards over sequences of states and allow performance measures 
to be evaluated on MCs through techniques for computing long run averages. 
Finally, in Q a temporal logic is used to directly specify performance measures 
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for Markovian transition systems, where the evaluation of such performance 
measures is conducted via model checking. 

Among the approaches mentioned above, in this paper we concentrate on that 
of p] . Its distinguishing feature is that of being deeply rooted in the Markovian 
process algebraic formalism. This has allowed us to define a performance measure 
sensitive congruence in the bisimulation style inspired by El, which permits to 
compositionally manipulate system specifications without altering the values of 
their performance measures. We recall that taking performance measures into 
account when e.g. compositionally minimizing the state space of a Markovian 
process algebraic specification is important. If for example the equivalence used 
in the minimization process gives rise to the merging of two states whose asso- 
ciated rewards are different, we come to the undesirable situation in which the 
original model and the minimized model result in two different values for the 
same performance measure. 

Following 110], the reward based Markovian bisimulation equivalence of 0 
considers two types of rewards: yield rewards, which are accumulated while stay- 
ing in the states, and bonus rewards, which are instantaneously gained when ex- 
ecuting the transitions. Such an equivalence essentially aggregates yield rewards 
whenever possible, but does not manipulate bonus rewards at all. 

The contribution of this paper is to extend the equivalence of 0 in order to 
aggregate as much as possible while retaining the bisimulation style, the con- 
gruence property, and the value of the performance measures. The extension is 
twofold. First, we show how to define a Markovian bisimulation congruence that 
aggregates bonus rewards as well. This is achieved by taking into account in the 
aggregation process the conditional execution probabilities of the transitions to 
which the bonus rewards are attached. Second, we show how to define a Marko- 
vian bisimulation congruence that allows yield rewards and bonus rewards to be 
used interchangeably up to suitable correcting factors, aiming at the introduc- 
tion of a normal form for rewards. More precisely, we demonstrate that this is 
possible in the continuous time case, while it is not possible in the discrete time 
case because compositionality is lost. 

Since the way in which bonus rewards can be aggregated is easy to find 
and the way in which yield and bonus rewards can be interchanged is known 
in the literature, this paper is especially concerned with investigating whether 
they respect compositionality or not. We also observe that, although such an 
investigation is conducted for a particular language (EMPAg^^ 0 ), its results 
can be applied to every Markovian process algebra. 

This paper is organized as follows. In Sect. 2 we recall the basic notions 
about reward structures. In Sect. 3 we recall EMPAgr^, a process algebra for the 
specification of continuous time and discrete time Markovian systems, and the 
related reward based Markovian bisimulation congruence. In Sect. 4 we present 
an improved reward based Markovian bisimulation equivalence that aggregates 
bonus rewards as well, we prove that it is a congruence, and we show a sound 
and complete axiomatization. In Sect. 5 we present a further improved reward 
based Markovian bisimulation equivalence that allows yield rewards and bonus 
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rewards to be used interchangeably in the continuous time case, we prove that 
it is a congruence, and we show a sound and complete axiomatization. In Sect. 6 
we report some concluding remarks. 

2 Reward Structures 

In the performance evaluation area, the technique of rewards is frequently used 
to specify and derive measures for system models whose underlying stochastic 
process is a MC. According to m, a reward structure for a MC is composed 
of: a yield function expressing the rate at which reward is accumulated 

at state i t time units after i was entered when the successor state is j, and 
a bonus function expressing the reward awarded upon exit from state i 

and subsequent entry into state j given that the holding time in state i was t 
time units. Since the generality of this structure is difficult to fully exploit due 
to the complexity of the resulting solution, the analysis is usually simplified by 
considering yield functions that do not depend on the time nor the successor 
state, as well as bonus functions that do not depend on the holding time of the 
previously occupied state: yij(t) = yi and bij{t) = bij. 

Several performance measures can be calculated by exploiting rewards. Ac- 
cording to the classifications proposed in P2E1, we have instant-of-time mea- 
sures, expressing the gain received at a particular time instant, and interval-of- 
time (or cumulative) measures, expressing the overall gain received over some 
time interval. Both kinds of measures can refer to stationary or transient state. 
In the following, we concentrate on instant-of-time performance measures. 

In the stationary case, instant-of-time performance measures quantify the 
long run gain received per unit of time. Given yield rewards yi and bonus re- 
wards bij for a certain MC, the corresponding stationary performance measure 
is computed as: 

^ 2/* • 7T, + ^ ^ ' 4'i,j (1) 

i i j 

where is the stationary probability of state i and (f>ij is the stationary fre- 
quency with which the transition from state i to state j is traversed, (pi^j is 
given by the stationary frequency with which state i is entered (i.e. the ratio of 
its stationary probability to its average holding time) multiplied by the proba- 
bility with which the transition from state i to state j is traversed given that 
the current state is i. In the case of a continuous time MC (CTMC) we have 
4’i,j = ■ Qi,j with Qij being the rate of the transition from i to j, while in the 

case of a discrete time MC (DTMC) we have cpij = tt^ • pij with pij being the 
probability of the transition from i to j. 

In the transient case, instant-of-time performance measures quantify the gain 
received at a specific time instant. Given yield rewards yi and bonus rewards 
bij for a certain MC, the corresponding transient state performance measure is 
computed as: 

i i j 



( 2 ) 
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where TTi(t) is the transient probability of being in state i at time t and 
is the transient frequency with which the transition from state i to state j is 
traversed at time t, which is computed in the same way as (pij with TTi(t) in 
place of TTi. 

3 An Overview of EMPA^r 

n 

EMPAgr^ |3] is a family of extended Markovian process algebras with generative- 
reactive synchronizations |S| whose actions are enriched in order to accommodate 
the specification of n S N different performance measures simultaneously. This 
is achieved by associating with every action a pair composed of a yield reward 
and a bonus reward for each performance measure. The number n is called the 
order. For the sake of simplicity, without loss of generality in this paper we deal 
with order 1 only. 

3.1 Syntax and Informal Semantics 

The main ingredients of our calculus are the actions, each composed of a type, 
a rate, and a sequence of n pairs of yield and bonus rewards, and the algebraic 
operators. As far as actions are concerned, based on their rates they are clas- 
sified into exponentially timed (rate A G R+ representing the parameter of the 
corresponding exponentially distributed duration), immediate (rate ooi^w to de- 
note a zero duration, with Z G N_|_ being a priority level and w G R+ being a 
weight associated with the action choice), and passive (rate to denote an 
unspecified duration with priority level I and weight w associated with the action 
choice). Moreover, based on their types, actions are classified into observable and 
invisible depending on whether they are different or equal to t, as usual. 

Definition 1. Let AType be the set of aetion types including the invisible type 
T, ARate = R+ U {00; ^, | I G N+ A w € R+} U {*i^w \ I G N+ A w € R+} be 
the set of aetion rates, ARew = R U {*} be the set of aetion rewards. We use 
a to range over AType, A to range over ARate, A to range over exponentially 
timed rates, A to range over active (i.e. nonpassive) rates, y to range over yield 
rewards (y if not *), and b to range over bonus rewards (b if not *). The set of 
actions of order 1 is defined by 

Acti = {<a. A, {y, b)> G AType x ARate x {ARew x ARew) \ 

(A G {*i,w I I G N+ Aw G R+} A y = 6 = *) V 

(A G R+ U { 00 / I I G N+ AwG R+} Ay,b G R)} ■ 

Definition 2. Let Const be a set of constants ranged over by A and let ATRFun 
= {ip : AType — > AType \ p~^{r) = {r}} be a set of action type relabeling 
functions ranged over by p. The set C\ of process terms of EMPAg„^ is generated 
by the following syntax 

R ::= 0 I <a. A, {yi,k)>.E \ EjL \ E[p] \ E + E\ E\\sE\A 
where L,S C AType — {r}. ■ 

The null term “0” is the term that cannot execute any action. 
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The action prefix operator “<a, A, {y, 6)>._” denotes the sequential composi- 
tion of an action and a term. Term <a, A, {y,b)>.E can execute an action with 
type a and rate A, thus making the corresponding state earn additional yield 
reward y and the related transition gain bonus reward b, and then behaves as 
term E. 

The functional abstraction operator “_/L” abstracts from the type of the 
actions. Term EjL behaves as term E except that the type a of each executed 
action is turned into r whenever a G L. 

The functional relabeling operator changes the type of the actions. 

Term E[(p] behaves as term E except that the type a of each executed action 
becomes 

The alternative composition operator expresses a choice between two 

terms. Term E1+E2 behaves as either term Ei or term E2 depending on whether 
an action of Ei or an action of E2 is executed. The choice among several enabled 
exponentially timed actions is solved according to the race policy, i.e. the fastest 
action is executed (this implies that each action has an execution probability 
proportional to its rate). If immediate actions are enabled as well, they take 
precedence over exponentially timed ones and the choice among them is solved 
according to the preselection policy: the immediate actions having the highest 
priority level are singled out, then each of them is given an execution probability 
proportional to its weight. The choice among several enabled passive actions is 
instead solved according to the reactive preselection policy: for each action type, 
the passive actions having the highest priority level are singled out, then each 
of them is given an execution probability proportional to its weight. Therefore, 
the choice among passive actions having different types and the choice between 
passive and active actions are nondeterministic. 

The parallel composition operator ||s expresses the concurrent execution 
of two terms. Term Ei \\s E2 asynchronously executes actions of E\ or E2 not 
belonging to S and synchronously executes actions of Ei and E2 belonging to S 
according to the two following synchronization disciplines. The synchronization 
discipline on action types establishes that two actions can synchronize if and 
only if they have the same observable type in S, which becomes the resulting 
type. Following the terminology of [^, the synchronization discipline on action 
rates is the generative-reactive mechanism, which establishes that two actions 
can synchronize if and only if at least one of them is passive (behaves reactively). 
In case of synchronization of an active action a having rate A executed by Ei 
{E2) with a passive action a having rate executed by E2 {Ei), the resulting 
active action a has rate/ weight given by the original rate/ weight multiplied 
by the probability that E2 (i?i) chooses the passive action at hand among its 
passive actions of type a. Instead, in case of synchronization of two passive 
actions a having rate and *12, W2 executed by E\ and E2, respectively, 

the resulting passive action of type a has priority level given by the maximum 
Imax between li and I2 and weight given by the probability that E\ and E2 
independently choose the two actions, multiplied by a normalization factor given 
by the overall weight of the passive actions of type a executable by Ei and E2 at 
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the priority level Imax- As far as rewards are concerned, since only the rewards 
of active actions are specified, in case of synchronization they are handled as 
follows. The yield rewards of an active action are treated exactly as the rate of 
that action, i.e. they are multiplied by the execution probabilities of the passive 
actions involved in the synchronization. Instead, the bonus rewards of an active 
action are just inherited, as multiplying them by the execution probabilities 
of the aforementioned passive actions would lead to an underestimation of the 
performance measures. The reason is that, in the calculation of the performance 
measures according to formulas m and 0, each bonus reward of a transition 
is multiplied by a factor that is proportional to the rate of the transition itself, 
hence multiplying the rates by the execution probabilities of passive actions is 
all we have to do. In the case of synchronization between two passive actions, 
the rewards of the resulting passive actions are still unspecified. 

Finally, we assume the existence of a set of constant defining equations of 
the form A = E. la. order to guarantee the correctness of recursive definitions, 
as usual we restrict ourselves to the set Qi of terms that are closed and guarded 
w.r.t. Defi- 



3.2 Reward Master-Slaves Transition Systems 



The semantic model of EMPAg^^ is a special kind of LTS called master-slaves 
transition system of order 1 (RMSTSi for short), whose transitions are labeled 
with elements of Acti. Recalling that active actions play the role of the masters 
(they behave generatively) while passive actions play the role of the slaves (they 
behave reactively), each state of a RMSTSi has a single master bundle composed 
of all the transitions labeled with an active action and, for each action type a, 
a single slave bundle of type a composed of all the transitions labeled with a 
passive action of type a. Since the operational semantics for EMPAgr,^ will be 
defined in such a way that lower priority active transitions are not pruned (in 
order to get a congruence) while lower priority passive transitions of a given type 
are, all the passive transitions belonging to the same slave bundle of a generated 
RMSTSi have the same priority level. 



Definition 3. A reward master-slaves transition system of order 1 (RMSTSi) 
is a triple 



{S,AType, ;►) 

where S is a set of states, AType is a set of aetion types, and 

> S Ai{S X Act I X S) is a multiset Q of transitions such that for all 

s € S and a £ AType: 



(s- 



^ s' A s ^ ^ s") ^ /' = I” 

A rooted reward master- slaves transition system of order 1 (RRMSTSi) is a 
quadruple 



{S, AType, ^,sq) 

where {S, AType, >■) is a RMSTSi and sq £ S is the initial state. ■ 

^ We use “II” and “|}” as brackets for multisets and M{S) (R(S')) to denote the 
collection of multisets over (subsets of) S. 
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We point out that the transition relation is a multiset, not a set. This allows 
the multiplicity of identically labeled transitions to be taken into account, which 
is necessary from the stochastic point of view. As an example, if a state has 
two transitions both labeled with <a. A, {y,b)>, using sets instead of multisets 
would reduce the two transitions into a single one with rate A, thus erroneously 
altering the average sojourn time in the state. 

Given a state, the choice among the bundles of transitions enabled in that 
state is nondeterministic. The choice of a transition within the master bundle 
is governed by the race policy if there are only exponentially timed transitions, 
the preselection policy if there are immediate transitions (which take precedence 
over exponentially timed transitions). The choice of a transition within a slave 
bundle of type a is governed by the preselection policy. 

We observe that the passive actions are seen as incomplete actions that must 
synchronize with active actions of the same type of another system component 
in order to form a complete system. Therefore, a fully specified system is per- 
formance closed, in the sense that it gives rise to a fully probabilistic transition 
system that does not include slave bundles. If in such a transition system we keep 
for each state only the highest priority transitions, then we can easily derive a 
performance model in the form of a reward DTMC or CTMC, depending on 
whether only immediate transitions occur or not. We point out that, if only im- 
mediate transitions occur, each of them is assumed to take one time step, hence 
the underlying stochastic model naturally turns out to be a DTMC. Should expo- 
nentially timed and immediate transitions coexist (in different states), a CTMC 
is derived by eliminating the immediate transitions and the related source states 
and by suitably splitting the exponentially timed transitions entering the re- 
moved source states, in such a way that they are caused to reach the target 
states of the removed immediate transitions. 

As far as the yield rewards are concerned, when constructing a reward MC 
from a RRMSTSi we proceed as follows. Whenever a state has several actions, be 
it due to an alternative composition operator or a parallel composition operator, 
we make the additivity assumption, i.e. we assume that the yield reward earned 
by the state is the sum of the yield rewards of its transitions. This assumption is 
consistent with the race inherent in the parallel composition operator and with 
the adoption of the race policy for the alternative composition operator, i.e. with 
viewing alternative actions as being in parallel execution, hence all contributing 
to the reward accumulation in the state. 



3.3 Operational Semantics 

The formal semantics for EMPAgr^ maps terms onto RRMSTSi. We preliminar- 
ily provide the following shorthands to make the definition of the operational 
semantic rules easier. 



Definition 4. Given a RMSTSi M = {S,AType, >■), s G S, and 

a G AType, we denote by La{s) the priority level of the slave transitions of type 
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a executable at s (La{s) = 0 if the slave bundle a of s is empty) and we de- 
note by Wa{s) the overall weight of the slave transitions of type a executable at s: 

Wa{s)=J2{\w\^s' GS.S 
Furthermore, we extend the real number multiplication to immediate rates as 
follows: 



The operational semantics for EMPAg^^ is the least RMSTSi 
{Qi, AType, >-i) satisfying the inference rules of Table ^ where in ad- 

dition to the rules (Chi;), (Ch2;), (Pal;), (Pa2;), (Syli) referring to a move of 
the lefthand process Pi, we consider also the symmetrical rules (Chl^), (Ch2r), 
(Pair), (Pa2r), (Sylr) taking into account the moves of the righthand process 
P 2 , obtained by exchanging the roles of terms Pi and P 2 . We consider the 
operational rules as generating a multiset of transitions (consistently with the 
definition of RMSTSi), where a transition has arity m if and only if it can be 
derived in m possible ways from the operational rules. 

Some explanations are now in order. First of all, the operational rules give rise 
to an interleaving semantics, which is made possible by the memoryless property 
of exponential distributions. The removal of lower priority passive transitions of 
the same type is carried out in rules (Ch2;) and (Ch2r) for the alternative 
composition operator and rules (Pal;) and (Pair) for the parallel composition 
operator by using La{E). 

In the case of a synchronization, the evaluation of the rate of the resulting 
action is carried out by rules (Pt/l;), {Sylr), and (Sy2) as follows. Whenever an 
active action synchronizes with a passive action of the same type, the rate of the 
resulting active action is evaluated in rules (Syli) and (Sylr) by multiplying the 
rate of the active action by the probability of choosing the passive action. The 
yield reward of the active action undergoes the same treatment, while the bonus 
reward is just inherited. Whenever two passive actions of type a synchronize, 
instead, the priority level and the weight of the resulting passive action are 
computed as described by rule (Sy2). In particular, the weight is computed by 
multiplying the probability p of independently choosing the two original actions 
by the normalization factor TV, which is given by the overall weight of the passive 
transitions of type a with maximum priority level executable by Pi and P 2 , 
computed by using Wa(P). 



Definition 5. The integrated semantics of E € Qi is the RRMSTSi 
Ii|P] = {Gi^e, AType, >i,e,E) 

where Gi,e is the set of terms reachable from E according to the RMSTSi 

{Gi, AType, ) and >i^e is the restriction of to transitions 

between terms in Gi,e- We say that E G Gi is performance closed if and only if 
Ii|P] does not contain passive transitions. We denote by £i the set of perfor- 
mance closed terms ofGi- ■ 

We conclude by recalling that from Ii|P] two projected semantic models can 
be obtained by essentially dropping action rates or action types, respectively. Be- 
fore applying such a transformation to Ii |P] , lower priority active transitions 
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Table 1. EMPAgr^ operational semantics 



a,\,(y,b) 

(Pv) <a,X,iy,b)>.E 



a,\,(y,b) 

E >1 E 

1) r, a ^ L 



a,\,(y,b) 

E/L >iE'/L 



a,X,(y,b) 

E >1 E 

(Hi 2 ) a e 



t,A,(s/,6) 

E/L >iE' /L 



(He) 



a,\,(y,b) 

E >iE' 



ip(a),\,(y,b) 

EVp] >1 E [if] 



aX,(y,b) 

El >1 El 



a,^,{y,b) 

El + E 2 >1 El 



(Ch2i) 



^1*1 

El ^ >1 E'l I > La{E2) 



El + E 2 E'l 



(Pal,) 



aX,{y,b) 

El >-1 El 



a,X,{y,b) 

El II s E2 >-1 E'l II s E2 



is 



El ^ >1 E'l I > La(E2) 

(P“ 2 ,) „ ,1 a ^ S 



a,*i Tjj ,(*,*) 

El II s E2 ^ >1 E'l II s E2 



a,X,{y,b) 

El >i E[ E 2 ^ >i E 2 

(a- “ M ^ -S' 



i,b) 



Wa{E 2 ) Wa{B 2 ) ’ 

Ei\\sE2 iEi\\sE2 



)(*»*) ^i*l2,W2 ’(*’*) 

El > 1 E'l E 2 > 1 E '2 

(Sy2) ^ ^ a€ S 



“.*max(ii,i 2 ).P-AT'(*.*) 

El II s E2 >■ 1 E'l II S E'2 



where: p = 



Wa(Ei) Wa(E 2 ) 



Wa{Ei) + Wa(E2) i!h=h 
N ^ Wa{El) it h > h 

Wa(E2) if I 2 > h 
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are pruned because E is no longer to be composed with other terms as it de- 
scribes the whole system we are interested in. The functional semantics 
is a standard LTS whose transitions are decorated with action types only. The 
Markovian semantics is instead a reward CTMC or DTMC, as seen in 

Sect. Id. 21 which is well defined only if E is performance closed. 

3.4 Reward Based Markovian Bisimulation Equivalence 

EMPAgrj^ is equipped with a reward based Markovian bisimulation equivalence, 
which relates two systems having the same functional, probabilistic, prioritized 
and exponentially timed behavior, as well as the same performance measure 
values, by considering their ability to simulate each other behavior. To achieve 
this, the rates/ weights of the transitions of the same type and priority level that 
leave the same state and reach states belonging to the same equivalence class are 
summed up, like in the exact aggregation for MCs known as ordinary lumping. 
The yield rewards labeling the transitions above are handled in the same way 
as the corresponding rates, because of the additivity assumption. The bonus 
rewards of the transitions above, instead, are not summed up, as this would result 
in an overestimation of the specified performance measures. The reason is that, 
in the calculation of the performance measures according to formulas (P) and ®, 
the bonus reward of a transition is multiplied by a factor that is proportional to 
the rate of the transition itself, hence summing rates up is all we have to do. 

Definition 6. We define function priority level PL : ARate — > % by: 

PL{*i,w) = -I 
PL{\) = 0 
PLi^ooi^-ufj — I 

and we extend the real number summation to rates of the same priority level 
and to unspecified rewards as follows: 

^l,Wi + *i ,W2 — ^l,Wi+W2 

^^l,W2 — ^^l,Wi+W2 
* * = * 

We define partial function aggregated rate-yield RY \ : Qi x AType x S x 
ARew X ViGi) ARate x ARew by: 

RY i{E, a, I, b, C) = (Ratei(E, a, I, b, C), Yieldi{E, a, I, 6, C)) 

where: 

Ratei{E,a,l,b,C) = Ell ^ I 3E' e C. E E' A PT(A) = / 1} 

Yieldi{E,a,l,b,C) = Ell ^ I 3A. 3E' G C. E E' A PT(A) = / 1} 
with RY i{E, a, I, b,C) = T whenever the multisets above are empty. ■ 



Definition 7. An equivalence relation B Q\ x Q\ is a RY-Markovian bisimu- 
lation of order 1 if and only if, whenever (Pi, E 2 ) G B, then for all a G AType, 
I G'Zi , b € ARew, and equivalence classes C G Qi/B 

RYi{Ei,a,l,b,C) = RY i{E 2 ,a,l,b,C) ■ 
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Table 2. Axiomatization of 




It is easy to see that the union of all the RY-Markovian bisimulations of order 1 is 
a RY-Markovian bisimulation of order 1. Such a union, denoted is called 

the RY-Markovian bisimulation equivalence of order 1. i® ^ congruence 
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w.r.t. all the algebraic operators as well as recursive constant definitions. In 
TableElwe report from 0 the sound and complete axiomatization of nonrecursive 
EMPAgr^ terms w.r.t. 

4 Aggregating Bonus Rewards 

As witnessed by axiom (^ 4 )^^, aggregates rates and yield rewards with- 

out manipulating bonus rewards at all. However, if we look at the way perfor- 
mance measures are computed according to formulas © and m, we note that 
whenever two actions are merged into a single one, then their bonus rewards 
can be aggregated as well. Unlike the rates and the yield rewards of those two 
actions, the bonus rewards are not just summed up as each of them needs to be 
preliminarily multiplied by the execution probability of the corresponding ac- 
tion. As an example, <a, Ai, {yi,bi)>.E -|- <a, A2, (j/2, b2)>-E can be equated to 
<a, Ai-kA 2 , {yi+y2, x^^-bi + .E-, similarly <a,coi^^^,{yi,bi)>.E+ 

<a,ooi^iu^,{y2,b2)>.E can be equated to <a,ooi^iui+w2,(yi + U2, ■ h + 

■ 62 )>■£'■ To be more precise, the probability by which each bonus reward 
involved in the aggregation must be multiplied is the probability of executing 
the corresponding action conditioned on the fact that one of the actions involved 
in the aggregation is executed. Considering such a conditional execution proba- 
bility instead of just the execution probability is not only necessary to preserve 
the value of the performance measures according to formulas © and (| 21 ), but 
is also crucial to get the congruence property. We introduce below an improved 
reward based Markovian bisimulation congruence that aggregates bonus rewards 
as well. 

Definition 8. We extend the real number division to rates of the same priority 
level as follows: 

j ^l,W 2 ~ ^l,Wi/w 2 

/ ^^l,W2 — W\/W2 

and we extend the real number multiplication to passive rates and unspecified 
rewards as follows: 

*i,w ■ * = * 

We define partial function aggregated rate-yield-bonus RYB\ : Qi x AType x 
% X V{Qi) -e^ARate x ARew x ARew by: 

RYBi{E, a, I, C) = (Ratei{E, a, I, C), Yieldi{E, a, I, C), Bonus\{E, a, Z, C)) 
where: 

Ratei{E,aJ,C) = Ell ^ I 6 . 3U' € C. U' A PL(A) = I R- 

Yieldi{E, a, I, C) = Ell ^ I 3A, b. 3E' e C. E' A PL(X) = I ^ 

Bonusi{E,a,l,C) = Ell Rate-,{E,a,i,c) ■ ^ I e C.U A 

pm = I 

with RYBi{E,a,l,C) = _L whenever the multisets above are empty. ■ 



148 



M. Bernardo and M. Bravetti 



Definition 9. An equivalence relation B Q\ x Q\ is a RYB-Markovian 
hisimulation of order 1 if and only if, whenever {Ei,E 2 ) € B, then for all 
a G AType, / G S , and equivalence classes C € Gi I B 

RYBi{Ei,a,l,C) = RYBi{E 2 ,a,l,C) ■ 

It is easy to see that the union of all the RYB-Markovian bisimulations of order 
1 is a RYB-Markovian bisimulation of order 1. Such a union, denoted ^j^j^ryb, 
is called the RYB-Markovian bisimulation equivalence of order 1. 

Theorem 1. ® congruence w.r.t. all the algebraic operators as well 

as recursive constant definitions. 

Proof. See ■ 



Theorem 2. Let be the set of axioms obtained from those in Table\^ by 

replacing (^ 4 )^'^ with 

<a,Ai,(yi,6i)>.^;-|- <a, A 2 , (^ 2 , &2)>-E = 

<a,\i + X2,{yi + y2,j^-h + j^^-b2)>.E 

if PL{Xi) = PL{\ 2 ). The deductive system Ded{Af^^) is sound and complete 
for over the set of nonrecursive terms of Gi • 

Proof See ■ 



Theorem 3 . Let E\,E2 G £\. If E\ ^2 then the value of the reward 

based performance measure is the same for E\ and E2. 

Proof. See m ■ 

5 Mixing Yield and Bonus Rewards 

Having the objective of defining a reward based Markovian bisimulation congru- 
ence that aggregates as much as possible, the question arises as to whether 
it is possible to consider just one type of reward instead of two. From 
the point of view of an equivalence, this can be rephrased in terms of be- 
ing able to jointly consider yield and bonus rewards. By looking at formu- 
las m and 0 and the way transition frequencies are computed, we note 
that in the continuous time case <a, Ai, (j/i, -I- <a, X2, {y2,b2)>-E can 

be equated to <a, Ai -I- A2, {yi + y2 + Xi ■ bi + X2 ■ 62, 0)>.£’, while in the dis- 
crete time case <a,ooi^wi,(yi,bi)>.E + <a,ooi^yj.^,{y 2 ,b 2 )>.E can be equated 
to <a, oo/,.u,i+u,2, (?/i + y2 + • bi + • &2, 0)>.F;. This gives rise to a 

normal form where only yield rewards are actually present, with bonus rewards 
being zero (or unspecified in the case of passive actions). However, in the dis- 
crete time case, we observe that the factor by which each bonus reward must 
be multiplied is equal to the execution probability of the transition to which it 
is attached. Since such a probability varies depending on the context in which 
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the term is placed, compositionality is lost. As an example, if we call Ei and 
E 2 the two equivalent terms above, respectively, and we take E^ defined by 
<a, oo/_i„3, (2/3, b'i)>.E, we have that the aggregated yield reward for Ei + E^ is 
2/1 + 2/2 + 2/3 + ■ bi + ^,+Zl+y., ■ b 2 + ■ bs, while the aggregated 

yield reward for E^ + E, is y, + y 2 + -h + • 62 + 2/3 + ‘ 0 + 

W 1 +ZI+W 3 ' bs ■ We introduce below a further improved reward based Markovian 
bisimulation congruence that mixes yield and bonus rewards in the continuous 
time case. 



Definition 10. We define partial function aggregated rate-reward 
RRi : QiX ATypexWi xV{Qi) -e^{ARatexARew)U{ARatexARewxARew) by: 

T)T) ( T? 1 n\ — I {Ratei{E, a,l,C), Reward i{E, a, C)) if 2 = 0 
KKi[E,a,l,C) - <^jiYBi{E,a,l,C) if 2 yf 0 



where: 



a,\,(y,b) 



Rewardi{E, a, C) = 2/ + A • 6 | 3E' G C. E >■ i E' A PL{X) = 0 |} 

with RRi{E,a,l,C) = _L whenever the multisets above are empty. 



Definition 11. An equivalence relation B C Qi x Qi is a RR-Markovian 
bisimulation of order 1 if and only if, whenever {Ei,E 2 ) G B, then for all 
a G AType, I gTL , and equivalence classes C G Gil B 

RRi{Ei,a,l,C) = RRi{E 2 ,a,l,C) ■ 



It is easy to see that the union of all the RR-Markovian bisimulations of order 
1 is a RR-Markovian bisimulation of order 1. Such a union, denoted i® 

called the RR-Markovian bisimulation equivalence of order 1. 

Theorem 4. ® congruence w.r.t. all the algebraic operators as well 

as recursive constant definitions. 

Proof. See B ■ 



Theorem 5. Let be the set of axioms obtained from by adding 

<“) A, {y, b)>.E = <a, X,{y-\-X-b, 0)>.E 

The deductive system Ded{A^^) is sound and complete for the set 

of nonrecursive terms of Gi . 

Proof See J^. ■ 



Theorem 6. Let E\,E 2 G £\. If E\ ~mb^^ E 2 then the value of the reward 
based performance measure is the same for Ei and E 2 . 



Proof. See 
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6 Conclusion 

In this paper we have improved the performance measure sensitive Markovian 
bisimulation congruence of 0 in order to aggregate more states and transitions 
while preserving compositionality and the values of the performance measures. 
While the congruence of 0 aggregates yield rewards without manipulating bonus 
rewards at all, the congruence of Def. E] aggregates also the bonus rewards, pro- 
vided that they are multiplied by the conditional probabilities of executing the 
actions to which they are attached, and allows yield rewards and bonus rewards 
to be used interchangeably in the continuous time case, provided that they are 
divided/multiplied by the rates of the actions to which they are attached. 

The impossibility result of this paper, i.e. the fact that it is not possible to 
define a performance measure sensitive Markovian bisimulation congruence that 
allows yield and bonus rewards to be used interchangeably in the discrete time 
case, emphasizes the necessity of the bonus rewards. In the literature of Markov 
reward processes it is well known that yield and bonus rewards can be used inter- 
changeably in the continuous time case, and in this paper we have verified that 
such a property does not violate compositionality. In the continuous time case 
the yield rewards work well because of the race policy. In particular, the additiv- 
ity assumption is sound because in every state all the transitions are viewed as 
being in parallel execution, hence each of them contributes with its yield reward 
to the accumulation of reward at the state. In the discrete time case, instead, the 
preselection policy applies, hence the bonus rewards are more natural to express 
performance measures. Besides being more convenient from the modeling view- 
point, in the discrete time case the bonus rewards are also necessary from the 
compositionality viewpoint, i.e. they cannot be transformed into yield rewards 
if we want to get a congruence. In fact, if we transform them into yield rewards, 
we have that the contribution of the transitions to the accumulation of reward 
at the state is given by their average bonus reward, i.e. the weighted sum of their 
bonus rewards with each of them multiplied by the execution probability of the 
corresponding transition. Since the above mentioned execution probabilities (un- 
like the rates in the continuous time case) vary depending on the environment 
in which the state is placed, compositionality is lost. 

The performance measure sensitive Markovian bisimulation congruence of 
Def. aggregates more than that of 0. The reason is that the new congru- 
ence can merge also those transitions that the old congruence cannot merge 
only because of their different bonus rewards. A quantification of the achieved 
improvement is left for future research. 

We conclude with a practice related observation. Describing the rewards di- 
rectly within the Markovian process algebra specifications of the systems has 
the advantage of allowing the specifications to be compositionally manipulated 
while preserving the values of the performance measures. This advantage on 
the analysis side is unfortunately diminished by a drawback on the modeling 
side: the system specifications are obfuscated with performance measure related 
details. However, the Markovian process algebra specification of a system and 
the specification of its reward based performance measures of interest can be 
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easily decoupled by separately describing the rewards to be attached to the ac- 
tions occurring in the system specification. A syntactical preprocessing step, like 
the one performed by the EMPAg^^ based software tool TwoTowers |3, then 
permits to automatically insert the rewards into the system specification. This 
avoids burdening the system specifications with rewards at modeling time, eases 
the specification of additional performance measures for the same system spec- 
ification, and allows every performance measure specification to be reused for 
different system specifications. 
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Abstract. In this paper, the concept of complete finite prefixes for pro- 
cess algebra expressions is extended to stochastic models. Events are 
supposed to happen after a delay that is determined by random vari- 
ables assigned to the preceding conditions. Max-plus algebra expressions 
are shown to provide an elegant notation for stochastic prefixes not con- 
taining any decisions. Eurthermore, they allow for the computation of 
performance measures. The derivation of the so called A;-th occurrence 
times is shown in detail. 



1 Introduction 

Stochastic process algebras (SPA) have become accepted languages for the de- 
scription of functional and quantitative aspects of distributed systems. Their 
compositionality allows for easy-to-read specifications and for the reuse of mod- 
ules. The evaluation and verification of SPA models is frequently based on in- 
terleaving semantics, which are prone to state-space explosion. Action delays 
are often restricted to exponential distributions, so that results can be obtained 
by Markovian analysis. Most interleaving semantics do not allow for general 
distributions. 

In this paper, we avoid some of the above restrictions by using a true- 
concurrency semantics: a finite stochastic event structure prefix. Because of its 
non-interleaving nature, it is generally smaller than an interleaving transition 
system, and has an explicit notion for the concurrent execution of actions. Ad- 
ditionally, it allows for general distributions for the description of action delays. 
Our semantics for a simple stochastic process algebra is based on the complete 
finite prefix for (non-stochastic) process algebra introduced in 0, which in turn 
was inspired by a McMillan prefix for Petri nets m (improved by (HI) . Expres- 
sions in max-plus algebra |2 are shown to be a natural description for timing 
properties of systems. We show that our approach is appropriate for the deriva- 
tion of performance measures. Similar work can be found in ISI 

The rest of this paper is organised as follows. In Section 2 we briefly review 
the derivation of a complete finite prefix for simple stochastic process algebra 
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expressions. Section 3 gives an overview of max-plus algebra, especially of max- 
plus matrix operations and their application to graphs and systems of linear 
equations. We use max-plus methods in Section 4 for the description and evalu- 
ation of prefixes. Section 5 concludes with a discussion of the results, drawbacks 
and further ideas. The material presented in this paper is based on 

2 Complete Finite Prefix for Stochastic Process Algebra 

This section gives a brief outline on computing prefixes. Further details can be 
found in IBS]- 

2.1 Stochastic Process Algebra 

We use a simple SPA throughout this paper. Its syntax is defined as follows. 

Definition 1. Let a he an action, F a distribution function with F{x) = 0 for 

X < 0, and A a set of actions. The stochastic process algebra expressions are 

defined by the following rules: 

P ::= stop I {a,F).P \ P + P \ P||^P | Id ^ 

stop is the process that does nothing. {a,F).P is the process that executes 
action a after a delay that is distributed according to F. We can think of a 
clock that is set to a randomly chosen delay (according to F) and that counts 
downwards to 0, where it eventually expires. After the clock has expired, action a 
can be executed. The execution of an action does not consume time. The process 
Pi + P 2 may behave either as Pi or as P 2 - Which behaviour is chosen depends on 
the clocks that are running for Pi and P 2 '. the fastest wins. If the winner can not 
be uniquely determined, the choice is made non-deterministically. The process 
Pi \\aP 2 describes the independent parallel execution of Pi and P 2 - Only if one 
of them wants to execute an action that is contained in the synchronisation set 
A, it has to wait until the other process becomes ready to execute this action as 
well. Both execute the action synchronously. An identifier (Id) is a place-holder 
for an expression P that is defined by an equation Id = P. We can instantiate 
a process several times, i.e., for recursive definitions. 

Example 1. As running example, we consider a simple buffer with a writing and 
a reading agent. Data of random length is written into the buffer (in) and later 
read from the buffer (out). In order to reflect the random length, both operations 
require an exponential delay, but reading is faster than writing. 

The writer thinks for a uniformly distributed amount of time (between 1 and 
5 time units) and writes then something into the buffer: 

Writer = (thinkW, unif orm([l, 5])).(in, exp(5)) .Writer 

The reader waits exactly 2 time units before it reads from the buffer: 

Reader = (thinkR, det(2)).(out, exp(8)). Reader 
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with 



Buffer = (in, det(0)).(out, det(0)).Buf f er. 
The entire system is described by 



System = Writer||{in}Buffer||{out}Rsader. 

Note that the buffer itself does not delay the Writer nor the Reader upon in 
and out actions, respectively, since its delay is 0 time units with probability 1. 
□ 



2.2 Condition Event Structures 

The base structure for prefixes are condition event structures. Events describe 
the possible occurrences of actions. Each possible occurrence of an action is 
denoted by an unique event. We will here identify events by their corresponding 
action name, possibly adding a subscript in order to form a unique name. 

Definition 2. A stochastic condition event structure is a tuple {D, E, U, ly^) 
where 

— D is a set of conditions, 

— E is a set of events, 

— G D X D is the choice relation (symmetric and irreflexive) 

<C {D X E) U {E X D) is the flow relation, 

— : D — > DF is a mapping from conditions to distribution functions. O 

Note that neither D nor E has to be finite. For technical reasons, we assume 
E to contain a bottom event T that denotes the start of the modelled system 
behaviour. 

Stochastic condition event structures have a simple graphical representation. 
Conditions are drawn as circles, labelled with their names and distributions. 
Events simply appear with their names. The flow relation is represented by 
arrows, the choice relation by lines with a U on them. 

An event structure models all possible behaviours of a system. With the start 
of the system, we also set imaginary clocks assigned to the conditions directly 
following the bottom event T, i.e., for all d G D : T ^ d. If for any event 
the clocks of all directly preceding conditions have expired, this event occurs 
immediately, starting the clocks of the succeeding conditions, and so on. 

The occurrence of an event e inhibits forever the occurrence of all events 
being in conflict with e. 

Downwards closed sets of pairwise non-conflicting events are called configu- 
rations. The local configuration [e] of an event e is defined by 

[e] := {e' G E\e' e}. 

0 is the local configuration of T. Corresponding to configurations are cuts: a cut is 
the set of conditions “following” a configuration. Each configuration corresponds 
to a cut and vice versa. After the occurrence of an event, usually some time has 
to pass before the next occurrence of an event. Consequently, cuts represent 
states in which the system may spend time. 
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2.3 Unfolding 

The algorithm in j0| for the construction of a condition event structure for a 
process algebra expression is derived from the unfolding algorithm for Petri nets 
m- So we refer to the resulting event structure as unfolding as well. As shown 
in p[j, the unfolding algorithm in jOj can directly be used to derive unfoldings 
for SPA expression, yielding stochastic event structures. 

The unfolding algorithm, which we do not describe in detail here, “un- 
folds” the SPA expression, generating conditions and events step by step. The 
conditions are additionally labelled with so-called components (via a mapping 
Ic)- Components are basically prefixed stochastic process algebra expressions, 
equipped with a notion for the synchronisation context in which they occur. 

Definition 3. Let P be a stochastic process algebra expression, a an action, and 
A a set of actions and F a distribution function. A component C is defined by 

C ::= stop I {a,F).P \ C|U I lUU. ^ 

Stochastic process algebra expression are decomposed into components (roughly 
by splitting parallel and choice expressions). 

Components are assigned to conditions. The choice relation between com- 
ponents, derived during the decomposition of SPA expressions, determines the 
choice relation between the respective conditions. 

For sets of components a structured operational semantics (SOS) can be 
defined |0|. Starting from a set of initial components, all possible transitions 
(labelled with actions) are derived according to this SOS. For each transition, a 
new event and its successor conditions (with appropriately assigned components) 
is introduced in the unfolding. Hence, each occurrence of an action is transformed 
into a unique event. The whole unfolding is created by successive application of 
this derivation of transitions from “unused” components in the unfolding. 

If an action is executed by a single component, the corresponding event will 
have exactly one preceding condition; if it is the result of a synchronisation, it will 
have several predecessors. Apart from the explicit choice, described by the choice 
relation on components, there can also be choice as a result of synchronisation on 
common labels that can be recognised by the existence of conditions with more 
than one succeeding event. In general (for recursive SPA expressions) there is an 
infinite number of possible occurrences of actions, and so the resulting stochastic 
condition event structure will be an infinite structure. 

States. A cut in the unfolding corresponds to a set of conditions, and so it 
does (via the mapping Ic) to a set of components. These sets will be such that 
they correspond to a valid stochastic process algebra expression that is reachable 
(via the classical interleaving semantics) from the original SPA expression. The 
expressions can be assembled by undoing the decomposition. They are called 
the states of the cut (resp. of the corresponding configuration). In ^ it is shown 
that exactly the reachable states are represented in the unfolding. 
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2.4 Prefix 

For finite state SPA processes, the possible behaviour is already represented in 
a finite prefix of the unfolding. In this section we comment on the construction 
of the prefix. 

Events are made comparable by a so-called adequate order IZ such that 
(roughly) e' Z e if e' is encountered earlier in the unfolding process. There 
is a local configuration [e] for all events e in the unfolding and consequently we 
have local states, written ^^([e]). We use the idea of states assigned to events for 
the construction of a finite prefix: Whenever we find an event e, such that there 
exists an event e' with e' Z e and <S't([e]) = 5't([e']) we call it a cutojf event. 
We do not consider the unfolding beyond the cuts of cutoff events. The resulting 
prefix is finite and complete (although not always the smallest possible). 

The events in the prefix can be seen as representatives of whole classes of 
events. They represent the finite number of possible actions, the occurrences of 
which determine the event classes. These classes may have an infinite number of 
elements. 

Example 2. In Figure ^ the unfolding and the complete finite prefix for the SPA 
expression System of Example Q is depicted. In order to distinguish different 
occurrences of an action, events have action names with unique subscripts. We 
name conditions with capital letters. □ 



3 Max-Plus Algebra 

3.1 Introduction 

The so-called max-plus algebra has been developed for the description and eval- 
uation of discrete-event systems (DES). A complete survey can be found in 
Stochastic DES are treated in ca. The max-plus algebra is an algebraic ring 
comprising the set of real numbers (extended by — oo) as carrier set and © and 
© as operations. ® is interpreted as max and © as + (we also write © for — ). In 
our models we express time by random variables that are mappings from some 
probability space into the set of real numbers. By 0 we denote the random vari- 
able which is — oo with probability 1; it is the zero-element for the maximum 
operation ®. With II we denote the random variable which is 0 with probability 
1; it is the unit-element for ©. 

3.2 Matrices and Weighted Directed Graphs 

The operations ® and © can be extended to vectors and matrices whose elements 
are drawn from the max-plus algebra. 

Definition 4. A directed graph (digraph) G = (V,E) consists of a set V = 
{1, 2, . . . , n} of nodes (vertices) and a set E C V x V of arcs ( edges). 

G is a max-plus weighted digraph, if it comes with a mapping w that assigns 
to each arc a max-plus random variable. <> 
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Fig. 1. Unfolding and Complete Finite Prefix for Writer I I in Buffer I I out Reader 



An alternative representation for a graph is an adjacency matrix A, where 

A ■= I ^ 

' ^0, otherwise. 

Of course it is also possible to construct a graph to a given nxn max-plus matrix. 
Squaring the adjacency matrix A results in A^, which represents the graph with 
an arc between two nodes rii and ri 2 into the resulting graph, if there is a path 
of length two in the graph described by A from node n\ to node U 2 - Its weight 
is given by the sum of the original weights. In the star of a matrix, paths of 
arbitrary length are subsumed. 

Definition 5. For A an adjacency matrix, A* is defined by the max-plus sum, 
that is the maximum, of all its powers: 

OO 

A* := 0A^ 

k=0 O 

In an acyclic graph, there is a maximum length for all paths, and so it is possible 
to compute the star with a finite number of operations: 

n 

for some n < oo. 
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3.3 Systems of Linear Equations 

We can formulate systems of linear equations in max-plus algebra, as we can 
do in the field of real numbers. The well-known techniques for the solution of 
systems of linear equations depend on the inverse of ©. Unfortunately, max 
has no inverse. Consequently, we are not able to solve general systems of linear 
equations in the max-plus algebra. However, there is a special case that is well 
suited for our purposes P|, as we will see in Section 0 

Theorem 1. Consider a system of linear equations as follows: T = T (g) A® S , 
where T is the solution vector we are looking for, A is a matrix and S a given 
vector of suitable dimensions. If A is strictly upper triangular, the solution is 
given by T = S' © H*. 

Proof. By insertion of S © in the equation T = T(©H©S. ^ 

4 Application to Prefixes 

We use the max-plus methods introduced in the previous section for the notation 
and evaluation of prefixes. 

4.1 Occurrence Times and Linear Max-Plus Expressions 

The flow relation of the event structure prefixes describes the causal depen- 
dencies between events. Conditions are equipped with random variables that 
describe the delay between events. Combining dependencies and delays, we can 
determine a random variable 0{e) describing the time the event e is bound to 
occur. 

We can derive these occurrence times recursively, starting with the bottom 
event T which we assume to happen at time 0(T) = 0=1. Then, we look at 
immediate successor events of T: their occurrence times are determined by the 
maximum delay assigned to the conditions between T and themselves (plus the 
occurrence time of T, which is zero). We introduce for each condition d a random 
variable X^i distributed according to lj^{d). Then we can express occurrence times 
for all events: we add the random variable of the appropriate conditions to the 
occurrence times of the predecessors and then take the maximum. 

Definition 6. The occurrence time 0{e) of an event e is defined recursively as 
0(T) := 1 and 0{e) := ® for e T, respectively. <> 

The resulting expressions are linear in the max-plus algebra. 

Example 3. For the unfolding in Figure Ewe have the following occurrence times: 
O(thinkWi) = 0(T) © Xa 
0 (thinkR 2 ) = 0(T) © Xc deterministic 

0 (in 3 ) = (O(thinkWi) © Xd) © (0(T) © Xb) 

0(thinkW5) = 0(in3) © Xp 

0(out4) = (0(in3) © Xc) © (0(thinkR2) © Xp) 

0(thinkR7) = 0(out4) © Xj 
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O(thinkWi) is a uniform random variable with expectation 3; 0(in3) is the 
sum of a uniform and an exponential random variable, its expectation is 3.2. 
0(thinkR7) is the sum of a deterministic random variable and the maximum of 
sums of random variables. Its expectation is 5.6857. □ 

4.2 Representation of Prefixes 

In their graphical form, prefixes have a close resemblance to max-plus weighted 
graphs. We now show how to construct a graph (and the corresponding matrix) 
for a stochastic event structure prefix. 

We interpret events as the nodes of a graph. The flow relation seen at event 
level represents the arcs, i.e., if there is a condition d such that ei A d A 62, then 
there is an arc from ei to 62, denoted (ei, 62). The weight of this arc is given by 
the delay assigned to d. 

Definition 7. For a finite stochastic condition event structure (I?, f, Zjp) 
the corresponding weighted digraph G is defined as follows: 

-V = F, 

— (ci, 62) ^ hd \ N y 3d € D ! e^ A d A 62 

and w((ei, 62 )) := Xd with X ~ h{d) . ^ 

Note that we are loosing information: the conflict between two events is no longer 
expressed! 

Example f. The graph (left) and matrix (right) of Figure El represent the prefix 
of Figure □ 

□ 



( 0 Xa Xc Xb 0 ^ 

00 0 Xd 0 

0 0 0 0 Xe 

0 0 0 0 Xq 

0 0 0 0 0 
\0 0 0 0 0 / 

Fig. 2. Graph and Matrix for the Prefix of Figured 




4.3 The Repeating Part 

In the preceding section we have shown how max-plus matrices can be used for 
the representation of prefixes, although they do not reflect the conflict situations 
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between events. Therefore, we have to restrict ourselves to decision-free prefixes, 
i.e., prefixes without conflicts between events; all events must eventually occur. 
For the corresponding SPA expressions this means that they are not allowed 
to contain the choice operator and that they have to avoid synchronisations on 
common labels. 

Finite prefixes are constructed by finding cutoff events in the unfolding of a 
stochastic process algebra expression. To any such cutoff event e in an unfolding 
there is another event e' that is smaller w.r.t. the chosen adequate order (e' C e) 
and has the same local state {St{e') = St{e)). As a consequence, for an event 
e*' succeeding the local configuration of e', there is an e* succeeding the local 
configuration of e and St{e*) = St{e*'). Thus, by repetition of this argument we 
can state that the events of the prefix between e' and e, i.e., the set [e] \ \e'], 
represents event classes with an infinite number of members. Since in decision- 
free systems all events must eventually happen, the occurrence of representatives 
of these classes are observed infinitely often in the evolution of the system. 

Definition 8. Let Cutoff be the set of all cutoff events in a given decision-free 
prefix (D, A, 0, /jr, /c) o,nd Cutoff’ the set of those preceding events that have 
the same local state. Then the set of repeating events of the prefix is defined as 

E^--= U N\ U [«']• 

Cutoff e'&Cutoff’ 

The set of repeating conditions is given by D’’ := {d € D \ 3e € E’’ : d ^ eV e ^ 
d}. The repeating part of the prefix is given by (£)’", A’’, 0, ^ [ {D’’ x U x 
£»’')). O 

Example 5. Figure 0(a) shows the repeating part of the prefix of Figure 0 □ 



4.4 The fc-th Occurrence Time 

When does the k-th representative of a certain infinite event class occur? As all 
time intervals in our system are described by random variables, the instant can 
also be expressed as a random variable. 

To make the formal treatment easier, we identify the events (or more precise, 
the event classes) E’’ of the repeating part with natural numbers starting with 
1, in such a way that < on the natural numbers respects the partial order on 
events. With respect to the graphical representation, we do a topological sorting 
of events; for the running example we actually have already done this with the 
subscripts added to action names. 

Definition 9. For event class i, Ti(k) is the instant of time when the k-th rep- 
resentative of this class occurs. Tfk) is then a max-plus vector containing the 
k-th occurrence times for all event classes. ^ 



® 

i 
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(a) First Instantiation 



(b) Second Instantiation 



Fig. 3. Repeating Part of the Prefix 



T(l) is easily determined. We simply take the original prefix and compute 
the occurrence times for all events in the repeating part. Put in the order given 
by the topological sorting, we obtain T(l). 

Theorem 2. Let ti he the minimal element of event elass i aeeording to -<* . 
The first oecurrence time of elass i is then given by the oecurrenee time of ei: 



®(1) = 0(e,). 



Proof. If 6i -<* e, then must occur before e. Because of being minimal, is 
the first representative of class i to happen. ^ 

For our example 

T(l) = {Xa,Xc,Xa®Xd,Xa®Xd®Xc®Xe). 

But how do we derive T(k) for A: > 1? We can append a new instantiation 
of the repeating part to the prefix by identifying (final) conditions of the prefix 
with equally labelled (initial) conditions of the repeating part (cf. Figure 01(b)). 
Doing this infinitely often, we obtain the whole infinite unfolding. 

The events of this new instantiation constitute the second occurrences of 
the infinite event classes. One can easily calculate their occurrence times. But 
there is a formal problem: we must distinguish between the random variables of 
conditions with identical labels from the first and the second instantiation. We 
do this by introducing explicitly the parameter k: Xd{k) denotes the random 
variable associated to d in the fc-th instantiation of the repeating part. 
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We already know T(l) when computing T(2). Similar to the derivation of 
ordinary occurrence times, we are able to describe Ti(2) by max-plus expressions 
concerning the direct predecessors of i. In general, Ti{2) depends on Tj{2) for 
its predecessor events j in the second instance of the repeating part. If any of 
the conditions preceding i is initial in the repeating part, Ti{2) depends also on 
T(l). For our example 

Ti(2) = T3(1) 0X^(2), 

T2(2) = T4(1)®Xc(2), 

T3(2) = Ti(2) 0 Xd{2) © T4(1) 0 Xb{2), 

T4(2) = T3(2) 0 Xg{2) © T2(2) 0 Xe{2). 



A sharp look reveals that this is a system of linear equations in the max-plus 
algebra. It can be rewritten in matrix form: 





/ 


/O0Ad(2) 0 \ 


\ 


/ 


/ 0 


0 0 o\\ 


T(2) = 


T(2)0 


0 0 0 Xe{2) 

0 0 0 Ag(2) 


© 


T(1)0 


0 

Xa{2) 


0 0 0 
0 0 0 




1 


1^0 0 0 0 / 


/ 


V 


\ 0 


Xc(2) Xb(2) & J J 



T(l) is already known, so the second part of the sum forms a vector, hence 
the equation is in the general form described in Theorem E 

The results for T(2) can be generalised. The vector T{k + 1) depends on itself 
for “internal” event classes, and on T{k) for “initial” event classes. 

Definition 10. The internal dependencies within the (fc + l)-th instantiation 

of the repeating part are expressed by 



Aij (k + 1) 



Xd{k+ 1), j, 

0, otherwise, 



The dependencies between the /c-th and the {k + l)-th instantiation of the 

repeating part are given by 



R (h — I ^d{k + 1), if d initial, d -< j, d' final, i -< d' , lc{d) = lc(d'), 
+ otherwise. 

With these definitions we get 

Theorem 3. 



T,(fc + 1)= |^0T,(fc+l)0Ay(/c+l)j © |^0T,(fc)0S„(A: + l) 
In matrix form: 



T{k + 1) = T(/c + 1) 0 A(fc + 1) © T{k) 0 B{k + 1) 



O 
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Due to the acyclic topology of the repeating part and the adequate labelling 
of event classes, the A{k) are strictly upper triangular matrices. Hence, according 
to Theorem n we state 

Theorem 4. 



T{k + 1) = T{k) ® B{k -b 1) 0 A{k + 1)*, for A: > 1. 

Proof. Application of Theorem^ ^ 

The calculation of T{k+1) now simply depends on T{k). Starting with T(l), 
which is given by the occurrence times, all T(k) can be determined recursively, 
step by step. For short, set C{k) := B{k)®A{k)*, such that the equation becomes 

T{k+l) :=T{k)®C{k + l). 

B{k) contains the dependencies of all event classes to the previous oc- 
currence of final events. A{k)* cumulates the internal delays such that they 
are expressed directly from one event to another without intermediate event. 
C{k) = B{k) 0 A(fc)* expresses the delay between the terminal events of instan- 
tiation number k and the {k + l)-th occurrence of all event classes. Thus, only 
the rows corresponding to terminal event classes contain entries different from 0 
at all. 



Example 6. For the running example, taking into account that Xsik) and Xc{k) 
are deterministically 0, we find: 



/ 



\ 



C{k) 



0 0 0 0 

XA{k) 0 XA{k)®XD{k) XA{k)®XD{k) 

\ 0 Xc{k) 0 Xc{k) (g) Xsik) J 



□ 



4.5 Cycle Time 

Based on the fc-th occurrence times of the event classes we can derive recurrence 
times for the different classes. As recurrence time we designate the time between 
the occurrence of two subsequent representatives of a single event class. It is 
again a random variable, describing the distribution of time. The recurrence 
times of all event classes are comprised in a vector RT{k): 

Definition 11. RTi{k) denotes the recurrence time between the k-th and the 
(k + l)th occurrence of an representative of event class i: 

RT{k) ■.= T{k+l)(gT{k) ^ 

The distributions of the recurrence times can change over time. We are interested 
in the long term behaviour of a system, so we take the limit of RT(k) as a measure 
for the general recurrence times (given existence): 

RT := lim RT(k). 

k—¥oo 
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The expected limiting recurrence time must be the same for all event classes. 
If one class were “faster” than the others, it would have to wait for its predeces- 
sors, so it would no longer be faster. Clearly this is only true for those prefixes 
with connected repeating part. 

We take this identical limiting recurrence times as cycle time for the whole 
system. It denotes the time it takes to execute one instantiation of the repeating 
part of the system. 

Definition 12. Let CT be the expected limiting recurrence time for any event 
class: 

CT := E[RTi]^ for arbitrarily chosen 1 < i < n. 

CT is called the mean cycle time of the system. 

Unfortunately, the calculation of the mean cycle time comes with all those 
problems known from the solution of task graph models . 

Example 7. For our example, the mean cycle time for all event classes is 

CT = E[{Xa ® Xd) © {Xc © Xe)] = 5.6857. 



□ 



4.6 Condition Holding Times 

So far we presented two simple performance measures that can be derived from 
a complete finite prefix: the fc-th occurrence times of events and the mean cycle 
time of the repeating part. In this section we show how to use the fc-th occur- 
rence time of events for the calculation of a performance measure concerning 
conditions. 

Conditions are assigned stochastic delays. If the system enters a condition, a 
clock is started according to the specified distribution function. When this clock 
reaches zero, the succeeding event is locally activated. But it does not need to 
be the case that the event is also globally activated. Sometimes the system must 
remain in a condition, even when the clock already has reached zero. The amount 
of time the system actually spends in a condition, i.e., the delay plus the possible 
waiting time, is called its holding time. 

Definition 13. Let a be a non-terminal condition of the repeating part. Then 

H (k\ = ^ 0 Tj2{k), if j2 

' \Tji{k) (Z)Tj 2 {k — if a initial, a ^ ji,j 2 ^ a',lc(a') = lc{a), 

denotes the holding time of condition a in the k-th instantiation of the repeating 
part. ^ 

In a decision-free unfolding, each condition has exactly one predecessor and 
one successor event. The holding time in the k-th instantiation of a condition i 
is then easily determined: it is the difference of the A:-th occurrence time of its 
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successor and the fc-th occurrence of its predecessor. If the condition is initial, 
the (fc — l)-th occurrence of the predecessor event has to be considered. 

The long-run holding time of a condition a is determined by the expectation 
of the limit of Ha(k), given existence: 



:=E 



lim Ha{k) 

k—^oo 



Example 8. Since ElA{k) = 3 for all k, EIa = 3. The sum of the mean holding 
times for A and D has to be equal to the mean cycle time: EIu = CT — EIa = 
5.6857 — 3 = 2.6857. Compare this to E[Xjj] = 0.2. The mean holding time for 
C is Elc = 2. Consequently, He = 3.6857 (compared with E[Xe] = 0.125). 

The writer is expected to think for three time units, then he is expected to 
wait for the buffer. This takes (on average) 2.4857 time units. Writing into the 
buffer takes 0.2 time units. For the reader process it is quite similar □ 

4.7 Stationary Probabilities 

Prefixes are a semantics for stochastic process algebra expressions. Those ex- 
pressions are composed of different agents, which interact with each other. The 
agents are further subdivided into components, which are then used as condition 
labels in the prefix. We can rebuild the agents from these labels, even if we only 
have the prefix. An agent is then represented by a set of conditions forming a 
chain w.r.t. the flow relation 

Example 9. In Fig. 0 the three agents of the repeating part of our example are 
depicted. □ 




Fig. 4. The agents (Writer, Buffer and Reader) of the repeating part 



For the repeating part of an agent we know the holding times of the corre- 
sponding conditions. Furthermore, we know the time the system needs for one 
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cycle. This is exactly the same time the agent needs for one cycle. Consequently, 
the sum of all holding times of one agent must be the cycle time. But this means 
that we are able to determine the part of time the agent spends in a particular 
condition. In other words, we can determine the stationary probability of a condi- 
tion. In contrast to stationary probabilities of Markov chains, these probabilities 
are local to the agent and do not concern the global system state. 

Definition 14. Let CT he the expected cycle time of a system and Ha the ex- 
pected holding time of condition a. Then the local stationary probability of 
condition a is given by the fraction 

_ ^ 

' CT' O 

Example 10. For the writer and the reader of the example, we obtain the fol- 
lowing local stationary probabilities: 

— The writer is thinking with probability tta = ^ = g ggg^ = 0.5276. He is 
waiting for or writing in the buffer with probability ttd = 0.4724. 

— The reader is thinking with probability ttc = 0.3518. He is waiting for or 
reading from the buffer with probability tte = 0.6482. 

5 Conclusion 

In this paper, we considered complete finite stochastic event structure prefixes 
as true-concurrency semantics for simple SPA expressions. After a short intro- 
duction in max-plus algebra, a method for the computation of occurrence times 
has been presented. Using the idea of cutoff events, we found the repeating event 
classes of a prefix. Similar to the calculation of occurrence times, fc-th occurrence 
times can be derived that are the basis for other performance measures. 

Max-plus algebra methods provide an elegant way to describe the timing 
behaviour of decision-free stochastic prefixes. The timing of repeating behaviour 
can be determined via the solution of a system of linear equations. Unfortunately, 
this solution is generally a complex task, as it involves sums and maxima of 
random variables, that even might be dependent. So, complexity may make 
max-plus matrices useful only for notational purposes. 

Finally, we have shown how some performance measures of SPA models with 
general distributions can be expressed in terms of the stochastic prefix and max- 
plus algebra. 
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Abstract. We consider a discrete time process algebra capable of (i) 
modeling processes with different probabilistic advancing speeds (mean 
number of actions executed per time unit), and (ii) expressing prob- 
abilistic external/internal choices and multiway synchronization. We 
show that, when evaluating steady state based performance measures 
expressed by associating rewards with actions, such a probabilistic ap- 
proach provides an exact solution even if advancing speeds are considered 
not to be probabilistic (i.e. actions of different processes have a different 
exact duration), without incurring in the state space explosion problem 
which arises with an intuitive application of a standard synchronous ap- 
proach. We then present a case study on multi-path routing showing the 
expressiveness of our calculus and that it makes it particularly easy to 
produce scalable specifications. 



1 Introduction 

The modeling experience in the specification of probabilistic concurrent systems 
(see, e.g., m and the references therein) has revealed the importance of using lan- 
guages expressing advancing speed of processes, probabilistic internal/external 
choices, and multi-way synchronization for representing the behavior of real sys- 
tems. In 0 we have considered a probabilistic calculus that combines, in a natu- 
ral way, these mechanisms. In particular, such a calculus adopts a mixture of the 
generative and reactive approaches by considering an asymmetric form of syn- 
chronization where a process which behaves generatively may only synchronize 
with processes which behave reactively. The integration of the generative and re- 
active approaches is naturally obtained, similarly as in H2|> by designating some 
actions, called generative actions, as predominant over the other ones, called 
reactive actions (denoted by a subscript “*”), and by imposing that generative 
actions can synchronize with reactive actions only. In particular, the parallel op- 
erator that we considered in P is similar to the CSP |0| operator P ||s Q, where 
processes P and Q are required to synchronize over actions of type in the set S 
and locally execute all the other actions. Such an operator expresses multi-way 
synchronizations by assuming that the result of the synchronization of a gener- 
ative action a and a reactive action a* is a generative action a, while the result 
of the synchronization of two reactive actions a* is again a reactive action a*. 
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As a consequence, an n-way synchronization is composed of all reactive actions 
except at most one generative action: the choice of a generative action a deter- 
mines the action type to be performed and the other processes internally react 
by independently choosing one of their reactive actions a,. Similarly as in |5j, our 
parallel operator is parameterized with a probability p, determining the process 
performing the next move in a term P ||g Q: we choose P with probability p 
while we choose Q with probability 1 — p. We call p and 1 — p the probabilis- 
tic advancing speeds of P and Q, respectively. As an example, let us consider 
a system composed of two sequential processes whose behavior is described by 
a term like0: (o -\-P b) ||'J(c -I-’’ d). According to 0, we first choose which of the 
two processes must make the next move according to probabilities q and 1 — q. 
Then, if the lefthand process wins we locally choose between a and b according 
to probabilities p and 1 — p, otherwise if the righthand process wins we locally 
choose between c and d according to probabilities r and 1 — r. Moreover, our 
approach integrates the generative-reactive approach inspired by H2| with the 
approach to probabilistic process choice inspired by P] in that the selection of 
the action to be executed in a system state is conceptually carried out through 
two steps (a generative choice determining the action type followed by a reactive 
choice), each possibly employing probabilistic choice among processes. Since we 
see the reactive actions as incomplete actions which must synchronize with gen- 
erative actions of another system component in order to form a complete system, 
fully specified systems always give rise to probabilistic transition systems which 
are purely generative. Fully specified systems are therefore fully probabilistic sys- 
tems (systems not including non-deterministic choices jHI), from which a Markov 
Chain can be trivially derived by discarding actions from transition labels. As a 
consequence, they can be easily analyzed to derive performance measures. 

In this paper we start by showing that, if we interpret probabilistic transi- 
tion systems produced with the calculus of P in a discrete time setting, where 
each transition takes a discrete time step to be executed, actions of processes 
are actually executed with the advancing speed expressed as parameters of the 
parallel operators. 

Differently from existing discrete time process algebras, where parallel pro- 
cesses are executed in synchronous locksteps (see, e.g., 1711 1181 1. the parallel 
operator that we adopt is asynchronous and allows processes with different prob- 
abilistic advancing speeds (mean number of actions executed per time unit) to 
be modeled. As we now show, P ||^ Q represents a system where the mean action 
frequency (number of actions executed per time unit) of process P is p, while 
the mean action frequency of process Q is 1 — p. Since P and Q may advance at 
different action frequencies, with respect to the classical synchronous approach, 
modeling a concurrent system does not necessarily imply adopting the same du- 
ration for the actions of P and Q. For instance, we could model a post office with 
a priority mail and an ordinary mail service simply by the term P Q, where: 
process P, representing the ordinary mail service, repeatedly executes actions a 
expressing the delivery of a letter via ordinary mail; process Q, representing the 



^ We use 11^ instead of ||g when 5 = 0. 
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priority mail service, repeatedly executes actions b expressing the delivery of a 
letter via priority mail. Supposed that we take minutes to be the time unit on 
which the post office specification is based, in P ||g-^ Q the mean frequency for 
the ordinary mail service is 0.2 letters per minute (288 per day) and the mean 
frequency for the priority mail service is 0.8 letters per minute (1152 per day). 
Therefore the actions of P take 5 minutes in the mean to be executed, while the 
actions of Q take 1 minute and 15 seconds in the mean to be executed. 

To be more precise the execution of a system P ||g Q is determined by as- 
suming a probabilistic scheduler that in each global state decides which process 
between P and Q will perform the next step. In particular P and Q advance in 
discrete steps and the scheduler decides who is going to perform the next move 
by tossing an unfair coin which gives “head” with probability p and “tail” with 
probability 1 — p. If the coin gives “head” P moves, if the coin gives “tail” Q 
moves. After a certain number, let us say n, of coin tosses, i.e. after n time units, 
the mean number of heads that have been extracted (steps P has made) is n ■ p 
while the mean number of tails that have been extracted (steps Q has made) is 
n ■ (1 — p). Formally such mean values are derived in the following way: n • p is 
the mean value of a discrete random variable following a binomial distribution 
with parameters n (number of experiments) and p (probability of success for 
each experiment). This means that P performs a mean of p steps per time unit 
and Q performs a mean of 1 — p steps per time unit. Hence p is P’s probabilistic 
advancing speed while 1 — p is Q’s probabilistic advancing speed. Note that in 
the semantics of P \ \^ Q we do not express an actual concurrent execution of 
the actions of P and Q (so that when time passes for an action of P then it 
passes also for a concurrent action of Q). The behavior of a system P ||g Q is, 
instead, described at a higher level of abstraction by a model with discrete time 
steps where there is no actual parallel execution, but only an interleaving of the 
steps of the two processes, where each step takes the same amount of time (a 
time unit). To make this more clear, we can interpret the semantics of PII5Q 
as originated by a single-processor machine executing both processes (P and 
Q) via a probabilistic scheduler. In this view choices in the global states of a 
system P ||g Q do not represent “races” between concurrent time delays (as it is 
usual for continuous time process algebras) but only probabilistic choices that 
determine which is the process performing the next discrete step. Therefore we 
do not assume memoryless distributed sojourn times as, e.g., in the continuous 
time model of H2!, but we simply execute a system transition every discrete time 
unit. In the execution of P ||g Q sometimes we perform a discrete move of P and 
sometimes we perform a discrete move of Q and what matters, from the perfor- 
mance behavior standpoint, is the frequency with which the actions of a given 
process are executed: even if we represent the system behavior at a certain level 
of abstraction where actions are not concurrently executed but just interleaved, 
we have that such a representation gives the correct execution frequencies for ac- 
tions of processes. Representing the system behavior just taking care of execution 
frequencies is the level of abstraction necessary in order to evaluate performance 
properties in discrete time. This because, since the states of a Discrete Time 
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Markov Chain (DTMC) are not endowed with different sojourn times (like e.g. 
in Continuous Time Markov Chains), the evaluation of performance measures 
in a DTMC is entirely based on the execution frequency of transitions. As we 
will see, the operator P\\^Q can be used both for modeling the execution of 
P and Q in a single processor machine by means of a probabilistic scheduler 
(strict interpretation of the semantics of P\\^Q), and for expressing, as in the 
post office example, the actual concurrent execution of processes P and Q whose 
actions have a (possibly) different mean duration (interpretation of the seman- 
tics of P llg Q as an abstract description of concurrency), depending on how we 
calculate the duration of the time unit to be considered for the composed system. 

In this paper we also consider the problem of modeling discrete time systems 
where the different advancing speeds at which concurrent processes proceed are 
not probabilistic. This means that if, e.g., the action frequency of a process P 
in a parallel composition is 1 /n, with n natural number, then each action of P 
takes exactly n time units to be executed. Through a standard approach based 
on a synchronous parallel composition, where parallel processes are executed in 
synchronous locksteps (see, e.g., imiii), processes executing actions with dif- 
ferent exact durations could be modeled as follows. Called np the duration of 
the actions of a process P (number of time units taken to execute each action 
of P), we compute the greater common divisor div of the set of durations np, 
where P is a process composing the system. Then for each process P we split 
each action in npj div subactions and we take div to be the action duration in 
the specification of the whole system. Such an approach has the problem that 
the state space of the system greatly increases due to the splitting of the ac- 
tions of processes. Our approach constitutes an approximated solution to this 
problem, in that action frequencies of processes are probabilistic instead of be- 
ing exact, but actions are not split. Nevertheless we show that, in the case of 
non-blocking processes (i.e. processes enabling at least a generative action in 
each state) while such an approximation may affect the performance behavior 
of the system during an initial transient evolution, it gives correct performance 
measures when the system reaches a steady behavior. Therefore as far as the 
evaluation of steady state based performance measures of systems is concerned 
(at least if they are expressible by associating rewards with actions 0), our 
approach avoids action splitting, hence the state space explosion problem, while 
preserving the possibility of exactly analysing concurrent processes with exact 
advancing speeds. 

Finally we present a case study which shows all the main features of our 
calculus. More precisely, we model and analyze an algebraic specification of a 
router implementing a probabilistic multi-path routing mechanism. This case 
study shows that: (i) our approach makes it possible to analyze systems whose 
components are specified through actions with largely different exact durations 
(in our model we have actions lasting from half a microsecond to 20 millisec- 
onds); (ii) expressing advancing speeds of processes via a parameterized parallel 
operator (instead of, e.g., via weights attached to the actions they can perform) 
is convenient from a modeling viewpoint because the modeler can first specify 
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the behavior of processes in isolation and then establish, independently on how 
they are specified, their advancing speed when composing them in parallel; (in) 
thanks to the use of our probabilistic parallel operator and to the generative- 
reactive mechanism, it is possible to define a specification of the router which is 
easily scalable to an arbitrary size of the routing table. 

The paper is organized as follows. In Sect. 2 we briefly recall the syntax of the 
calculus of In Sect. 3 we show how to calculate the time unit to be considered 
for the composed system depending on the interpretation (processes executed by 
a single processor machine or actually concurrent processes) given to the parallel 
operator. In Sect. 4 we show that through our approach we can model systems 
where actions of different processes have a different exact duration. Finally, in 
Sect. 5 we present the case study on multi-path routing. 



2 Syntax of the Calculus 

In this section we briefly recall the generative-reactive calculus introduced in • 
Formally, we denote the set of action types by AType, ranged over by a, b, . . .. 
As usual AType includes the special type t denoting internal actions. We denote 
the set of reactive actions by RAct = {a, | a G AType} and the set of generative 
actions by GAct = AType. The set of actions is denoted by Act = RAct U GAct, 
ranged over by tt, tt', . . .. The syntax of the generative-reactive process algebra 
is defined as follows. Let Gonst be a set of constants, ranged over by A,B,.. .. 
The set C of process terms is generated by the syntax: 

P y.^OlTr.PlP+P P\P\\Pp\P[a^b]P\A 

with a G AType — {t}, b G AType, S C AType — {t}, p g]0, 1[. The set C is ranged 

over by P, Q, We denote by G the set of guarded and closed terms of £. 

We use the following abbreviations to denote the relabeling of several action 
types. Let “P[ip}\ where is a finite sequence ((oi, 61 )^^ • ■ • ? (on, of pairs 

of actions (at, bi) such that Ui ^ r, with an associated probability pi, stand for the 
expression P[a\ — >■ b\]P^ . . . [a„ — >■ relabeling the actions of type oi, . . . , a„ 

into the visible actions 5i, . . . , 6 „, respectively. For the sake of simplicity, we omit 
probabilistic parameters in the operators of our calculus whenever they are not 
meaningful. For a presentation of the semantics of the calculus we refer to HI2|. 

3 Expressiug Processes with Differeut Actiou Duratious 

In this section we show how our probabilistic parallel composition operator 
P llg Q can be used to express the concurrent execution of processes P and Q 
whose actions have a different duration. If we strictly follow this single-processor 
interpretation of the semantics of P\\^Q, we assume that the specifications of 
processes P and Q are based on the same time unit u representing action dura- 
tion, and consequently that actions of P ||g Q also take time u to be executed, 
i.e. u is also the time unit of P ||g Q. Since P and Q must share a single resource 
(the processor), the effect of putting P in parallel with Q is that both P and Q 
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get slowed down. In particular, when P (Q) is considered in isolation it executes 
one action per time unit u, when, instead, it is assumed to be in parallel with Q 
(P) by means of P Q, it executes p {1 — p) actions per time unit u. 

On the other hand such an action interleaving based representation of P\\^Q 
can be interpreted as being an abstract description of the actual concurrent 
execution of two processes P and Q specified with respect to (possibly) different 
action durations, as in the case of the post office example of Sect. 1. In general, if 
fp is the mean action frequency assumed in the specification of P (an action of 
P is assumed to take time 1/ fp in the mean to be executed) and /q is the mean 
action frequency assumed for the specification of Q, it is easy to derive a time 
unit u and a probability p such that, if we assume actions to take time u to be 
executed, P\\^Q represents the actual concurrent execution of processes P and 
Q with mean action frequencies fp and /g, respectively. Since the mean action 
frequency of process P is fp and the mean action frequency of process Q is /g, 
the mean action frequency of P \\^ Q must he f = fp + /g. Therefore the time 
unit representing the duration of the actions of P\\^Q that we have to consider 
is u = l// = 1/(/p + /q)j and the action frequency p of P with respect to the 
new time unit u is given hy fp = p/u = p ■ f, hence p = fp/ f = fp/ {fp + /g). 
Similarly the action frequency 1 — p of Q with respect to u turns out to be 
I — p = fq/f = fq/ifp + /g)- It is worth noting that by adopting a suitable 
time unit in this way, the speed at which P and Q are executed when they are 
considered in isolation is not reduced when they are executed in parallel. 



4 Expressing Processes with Exact Advancing Speeds 

In this section we show that when evaluating steady state based performance 
measures, we are able to deal with processes proceeding with different advancing 
speeds which are not probabilistic. In particular, while during an initial transient 
evolution considering the action frequency of processes as being exact instead of 
probabilistic may lead to different results when evaluating performance, in the 
case of non-blocking processes this does not happen when the system reaches a 
limiting steady behavior. 

As already explained in Sect. 1, executing a process P in parallel with another 
process which proceeds with a different action frequency, could be done through 
a standard approach based on a synchronous parallel composition by adequately 
scaling the time unit on which the specification of P is based, i.e. by splitting 
each action of P in a certain number n of subactions (see | 2 | for the details). 

Example 1. Let us consider a communication system composed of a process- 
ing unit receiving messages from an incoming channel and, after some internal 
computation, sending out them to an outgoing channel. Suppose that P is a 
specification of such a system where the time unit is considered to be a second, 
i.e. an action takes one second to be executed, and P is the underlying DTMC. 
Suppose that we want to evaluate the throughput of the system in terms of the 
number of messages sent out per time unit. In order to do this we consider a 
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reward structure, where we associate a reward equal to 1 to each action repre- 
senting the sending of a message and a reward equal to 0 to each other action 
of the system specification. Now, let us suppose that we want to express the be- 
havior of the system with respect to a different time unit, e.g. tenth of seconds 
instead of seconds. We may need to do this because we want to execute it in 
parallel with another process whose specification is made in tenth of seconds. 
Scaling the time unit by a factor 1/10 can be made by splitting each action a of 
P in 10 subactions (9 idle actions followed by action a), thus obtaining a scaled 
DTMC P'. The reward structure that we consider for the time scaled system 
is unchanged with respect to the reward structure considered for the original 
system. This because the reward gained by actions is not related with their du- 
ration but it is just used to count the occurrences of the actions, i.e. rewards 
0 and 1 associated to actions are not durations expressed in seconds, but just 
numbers. By calculating the throughput of the time scaled system (see 0), we 
obtain mt/10, i.e. one tenth of the throughput mt of the original system. This 
is an expected result because the throughput is a frequency which is expressed 
in number of actions executed per time unit and we changed the time unit from 
seconds to tenth of seconds. 

Now let us suppose that we want to evaluate the utilization of the processing 
unit in terms of the percentage of time occupied by the system in performing 
internal computations for message processing. We consider a reward structure 
where we associate a reward 1 to each action representing an internal compu- 
tation of the processing unit and a reward equal to 0 to each other action of 
the system specification. Now, similarly as in the previous case, let us suppose 
that we want to scale the time unit by a factor 1/10. Considered the DTMC 
P' obtained by scaling P, we have to evaluate the reward structure to be as- 
sociated with the time scaled system. Differently from the case of the message 
throughput, the reward gained by an action is related to the duration of the 
activity it represents, i.e. rewards 0 and 1 are durations expressed in seconds. 
Hence, when we consider as the time unit tenth of seconds instead of seconds 
each reward must be multiplied by 10. By calculating the utilization of the time 
scaled system (see |3), we obtain the same utilization of the original system. 
This is an expected result because the percentage of utilization of the processing 
unit does not change if we scale the time unit for each activity of the system. □ 

By employing our probabilistic parallel composition operator we can approx- 
imate the scaling of a factor 1/n of the time unit used in a specification P 
by executing P with a probabilistic action frequency 1/n. This is obtained by 

considering, e.g., the term P Idle, where Idle = idle. Idle and p = 1/n. 

Theorem 1. The steady state based performance measures o/Pjlg/dZe express- 
ible by attaching rewards to the generative actions of P are axactly as those 
derived by executing the generative actions of P with an exact frequency p.^ □ 

^ In this theorem and in the following TheoremElwe consider performance measures of 
periodic DTMCs to be evaluated from their time averaged steady state probabilities. 
The proof of both theorems can be found in |5]. 
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Example 2. Let us consider the communication system P of Example ^ where 
the time unit is taken to be one second and P is the DTMC underlying P. Now 
we see how to evaluate the two performance measures of Example ^by employing 
our approach based on probabilistic parallel composition. We scale the time unit 
of P to tenth of seconds by executing P with a probabilistic action frequency 
1/10. In particular we consider the term P Idle and its underlying DTMC 
P'. Both in the case of the system throughput and in the case of the processing 
unit utilization (by using the same reward structure for the time scaled system 
as that considered in Example ^ , we obtain the same performance measures as 
with the approach based on action splitting (see |2|). □ 

It is worth noting that, with respect to the standard approach based on action 
splitting, scaling the time unit via our probabilistic parallel operator gives the 
new possibility of using scaling factors p which are not of the form p = 1/n with n 
natural number. In general P \\^ Idle, when considered at the steady state, scales 
the time unit up used in the specification of P of a factor p: from 1/up = p/u we 
derive u = up -p where u represents action duration in the behavior of P Idle. 

Finally, we have that with our probabilistic parallel operator we obtain a 
correct time unit scaling at the steady state also for non-blocking processes 
which are part of a larger system. Hence we can express the parallel composition 
of processes specified with respect to different time units through a common time 
unit by considering a different scaling factor for the time unit of each process. 

Theorem 2. Supposed that both P and Q never block during the execution of 
P \\^ Q (in all the states of P\\^Q at least one generative action of P and one 
generative action of Q are executable), the steady state based performance mea- 
sures of P\\^Q expressible by attaching rewards to the generative actions of P 
and Q are axactly as those derived by executing the generative actions of P and 
Q with an exact frequency p and 1 — p, respectively. □ 

Therefore given two different time units up and uq representing action duration 
in the specifications of P and Q in isolation, respectively, then there exists a 
common time unit u and a probability p (determined from fp = 1/up and 
/q = l/uQ as explained in Sect. 3) such that P ||g Q just expresses the concurrent 
execution of the two processes without affecting their advancing speed. 

5 A Case Study: Multi-path Routing 

In this section we present a case study showing how our approach provides a 
compositional and intuitive method for specifying concurrent systems in a scal- 
able way. In particular, we consider a multi-path routing mechanism of the OSI 
network layer m, and we model and analyze an internetworking node (termed 
Interface Message Processor, IMP for short), whose arriving packets have several 
possible destinations with several possible ways to reach a destination. 

The routing algorithm decides, at the network layer, on which output link 
an arriving packet should be sent, depending on the destination of that packet. 
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We abstract from the algorithm used to determine an optimal path between two 
nodes of a network, and we assume that the modeled IMP has a routing table in- 
cluding the route information with several possible choices for each destination. 
A weight is associated to each possible path and these weights are used as proba- 
bilities to decide where to send the present packet. Supporting multiple paths to 
the same destination, unlike single-path algorithms, permits traffic multiplexing 
over multiple lines and substantially provides better throughput and reliability. 

5.1 Algebraic Specification of the Multi-path Router 

The overall model of our IMP (term Multipath) is shown in Table Ein the case 
of two possible destinations (a and b) and two possible paths for each destina- 
tion (oi, 02 for o, and bi, 62 for b). The algebraic specification is composed of 
several processes which are actually concurrent and are specified with respect 
to different time units. In particular, system Multipath consists of three concur- 
rent components: a term Arrivals modeling the incoming traffic, a term Router 
modeling the core of the IMP, and a term Channels modeling the outgoing chan- 
nels. The structure of the three components is as follows: (?) Term Arrivals is 
composed of two concurrent processes Arrivala and Arrivalb which model the 
incoming traffic directed to destinations a and b, respectively. The time unit 
representing action duration that we consider for both processes is one millisec- 
ond. The adoption of such a time unit makes it easy to represent a realistic 
workload for the IMP. In particular we assume that for each destination at most 
one packet per millisecond can arrive to the IMP, i.e. the maximum frequency 
of the incoming traffic is 2000 packets per second, (ii) Term Router represents a 
process whose time unit is half a microsecond, i.e. it can execute 2000000 actions 
per second. As we will see, the Router term, which is the core of the IMP, is a 
single processor machine managing the packets directed to the two destinations 
via a probabilistic scheduler. (Hi) Term Channels is composed of four concurrent 
processes modeling the possible channels oi, 02, 61, and 62- Since we take packet 
transmission to be represented by the execution of a corresponding action, their 
time units are defined on the basis of their bandwidth. The time unit for the 
channel a\ is 10 milliseconds, i.e. it can send out 100 packets per second, while 
the time unit for the channel 02 is 2.5 milliseconds (400 packets per second). 
The time units for the channels 61 and 62 are 5 milliseconds (200 packets per 
second) and 4 milliseconds (250 packets per second), respectively. 

Note that it is possible to compose in parallel the above processes, which 
are specified with respect to different time units, because, as it can be eas- 
ily verified, each of them never blocks during system execution. In order to 
express the actual concurrent execution of such processes, all the time units 
used in their specification are scaled to a common global time unit u. More 
precisely, u is evaluated by computing the inverse of the global action fre- 
quency of the composed system. Hence, in our case study u is the inverse of 
1000-1-1000-1-20000004-100-1-400-1-200-1-250 actions per second, i.e. u = 1/2002950 
seconds. Such a global time unit u determines the action frequencies to be consid- 
ered as parameters of the parallel operators used to describe the concurrent ex- 
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Table 1 . Multi-path Routing Model 



Multipath 

S 

C 

Arrivals 

Channels 

Router 

Si 

Queues 

Switch 

S’2 

B' 

v' 

n 



Arrival a 
Arrivalb 
Queue 
Queue' 
Manager 
Manager' 
Routing 
Routing' 
Idle 
Channel 



= Arrivals {Router Channels) 

= {receivea, receiveb} 

= {avaiLchai, avaiCcha2, avaiCchbi, 

avail-chb2, transmai, transma2, transmbi, transmb2} 

= Arrivala || 2 Arrivalb 

= {Channel[ip'i] ||^“ Channellip'2]) H” {Channel\(p'{] Channel[(p'2]) 

= {Queues ||siur Switch) ||/ Idle 
= {accepta, acceptb} I = {idle} 

= Queue[ifi'] ||/ Queue[<fi"] 

^ {Manager [(p']\\s'^{Routing[ifi'][(fi'i] \\}^, Routing[gi'][(fi'2]) ) ||f 
{Manager[ip"] \\si^{Routing[<p"][g>'{] W^g,, Routing[ip"][(p '{]) ) 

= {senda, busya] S'2 = {sendb, busyb} 

= {busya} B" = {busyb} 

= {{receive, receivea), {accept, accepta), {send, senda), {busy, busya)) 
= {{receive, receiveb), {accept, acceptb), {send, sendb), {busy, busyb)) 
= {{transm, transmai), {avaiCch, avaiLchai)) i £ { 1 , 2 } 

= {{transm, transmbi), {avaiLch, avaiLchbi)) i £ { 1 , 2 } 

= receivea. Arrivala wait. Arrivala 

= receiveb. Arrivalb -I-’'*’ wait. Arrivalb 
= receive,. Queue' + idle,. Queue 
= receive,. accept^. Queue' + accept Queue 
= accept. Manager' + idle,. Manager 
= send. Manager + busy .Manager' 

= send,. Routing' + avaiLch,. Routing 
= transm,. Routing + busy ^.Routing' 

= idle. Idle 

= avaiLch. Channel -|- transm. Channel 



ecution of system components in Tabled In particular, parameter pi represent- 
ing the advancing speed of term Arrivals in Arrivals {Router ||^ Channels) 
is given by the ratio of the action frequency of term Arrivals over the global 
action frequency of system Multipath, i.e. if we express action frequency in 
seconds pi = 2000/2002950 ~ 0.000999. Similarly, parameter p 2 representing 
the advancing speed of term Router in Router Channels is given by the ra- 
tio of the action frequency of term Router over the global action frequency 
of term Router \\q Channels, i.e. p 2 = 2000000/(2000000 -I- 950) ~ 0.999525. 
As far as the specification of the Arrivals component is concerned, the par- 
allel composition of the two concurrent processes Arrivala and Arrivalb has 
parameter | because their action frequency is the same (1000 actions per sec- 
ond). As far as the specification of the Channels component is concerned, in 
{Channel[(p'i\ ||p“ Channel[ip' 2 \) ||’' {Channel[ip'{] Channel[p' 2 ]) we take: pa 
to be 100/(100 -h 400) = 0.2; pb to be 200/(200 -h 250) 0.444444; and v to be 



178 



M. Bravetti and A. Aldini 



500/(500 + 450) « 0.526316. As an example of the calculated action frequencies, 
process Arrivala executes pi • | « 0.000499 actions per time unit u, i.e. 1000 
actions per second, and process Router executes (1 — Pi) ■ P 2 ~ 0.998527 actions 
per time unit u, i.e. 2000000 actions per second. 

Now let us describe in detail the behavior of each process of the system: (i) 
Process Arrivala (Arrivalb) models the incoming traffic through a Bernoulli dis- 
tribution with parameter ra (rb). In particular, an arriving packet is represented 
by the action receive a (receiveb) which synchronizes with the corresponding re- 
active action in the queue for packets a (6) of term Router. In the case such a 
queue is full the action receivea (receiveb) is not enabled and the arriving packets 
are lost (the generative action wait is executed with probability 1). (ii) Process 
Router is the core of the IMP and is composed of a term Queues collecting 
the arriving packets, a term Switch delivering the packets to the outgoing chan- 
nels, and a term Idle modeling the phases of router inactivity. They are defined 
as follows. Term Queues consists of two Queue processes, one for each kind of 
packet, which behave reactively. In particular, they receive packets destined to 
a (b) through reactive actions of type receivea (receiveb) and pass them to the 
Switch term through reactive actions of type accepta (acceptb). For the sake 
of simplicity we assume both queues to be of size 2. Term Switch is a single- 
processor machine executing two different terms, each one managing packets 
with a certain destination (a or 6), via a probabilistic scheduler with parameter 
p. In this way, by varying p, we can model an IMP that delivers packets with 
a particular destination more efficiently than packets with another destination, 
e.g. for commercial reasons. The term delivering packets to destination a (b) is 
composed of a Manager and two Routing terms, each one delivering packets to a 
particular channel Oi or 02 (&i or 62)- Term Manager accepts packets destined to 
a (b) from the dedicated queue through action accepta (acceptb) and afterwards 
either immediately passes them to one of the two Routing terms through action 
senda (sendb), or waits until at least one channel is available for transmission by 
performing action busy a (busyb). This behavior is realized through a generative- 
reactive mechanism as follows. The two Routing terms behave reactively and 
each of them accepts packets through a reactive action of type senda (sendb) 
and transmits them through the corresponding channel via a reactive action of 
type transma (transmb). Whenever the generative action senda (sendb) is en- 
abled by the Manager term, the Routing term accepting the packet is chosen 
according to the probability qa (qb) parameterizing the parallel composition of 
the two Routing terms. Note that a Routing term may be not available for ac- 
cepting a packet because it is currently transmitting through action transma 
(transmb) a packet previously received. Therefore, whenever only one Routing 
process is available for accepting a packet coming from the Manager term, the 
packet is transmitted through the corresponding channel with probability 1. 
Whenever both Routing processes are busy, the transmission of packets destined 
to a (b) is not possible and this is signalled to the Manager term through a 
multiway synchronization by enabling the reactive action of type busy a (busyb). 
Term Idle executes an action idle (representing the fact that the IMP is idle) 
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whenever term Router has nothing else to do. More precisely, action idle is 
executed through a multiway synchronization with all the other Router com- 
ponents if and only if the input queues (term Queues) are empty and the core 
of the IMP (term Switch) is not waiting for delivering a packet to the channel. 
In particular, term Idle prevents the term Router from blocking, thus allowing 
the advancing speed of terms Arrivals , Router, and Channels to be preserved 
and satisfying the condition needed for composing processes with different time 
units, (in) The four Channel processes model the outgoing channels a\, 02 , b\ 
and 62 • Each process Channel can either be transmitting a packet when the gen- 
erative action transm is synchronized with the corresponding reactive action of 
term Routing managing that channel, or be available for transmission when the 
generative action avaiCch is synchronized with the corresponding reactive action 
of term Routing. For instance in the case of channel oi the generative actions 
transmai and avail_chai must synchronize with the reactive actions transmai^ 
and avail -chai^ of the Routing term managing channel ai, respectively. In this 
way the generative actions of a Channel process are executed in mutual exclu- 
sion in the sense that in every system state one and only one of them is enabled. 
As a consequence term Channels never blocks. 

Thanks to our approach which allows processes specified with respect to 
different time units to be modeled without splitting actions, we have that the 
transition system underlying the algebraic specification of Table Q is composed 
of 576 states and 4768 transitions only. This is a crucial result, because if we 
model the same system by resorting to a classical approach which scales the 
time unit by splitting each action, we have to cope with the serious problem 
of a greatly increased size of the state space. For instance, since the basic time 
unit for the router is half a microsecond while the basic time unit for the in- 
put channels is a millisecond, in order to compose in parallel terms Arrivals 
and Router we have to split the actions of term Arrivals in thousands of sub- 
actions thus causing a state space explosion. Moreover, we point out that the 
generative-reactive behavior of the Switch process represents the core of this case 
study. In particular, process Switch generatively decides, according to probabil- 
ity p, which of the two Manager terms performs a send action {senda for the 
manager delivering packets to destination a or sendb for the other one), while 
it reactively decides, according to probability qa or qb (depending on the send 
action performed) which of the terms Routing synchronizes with such an action. 
A calculus capable of expressing generative-reactive choices is, therefore, very 
suitable (if not necessary) to model systems with such a behavior. Finally, it is 
worth noting that, thanks to the choice of putting probabilities in the operators 
instead of, e.g., attaching them to actions, and to the expressive power of our 
generative-reactive approach, it was possible to specify the IMP in such a way 
that all the probabilistic mechanisms on which its behavior is based (and which 
are not related with the internal behavior of a process) depend on the parame- 
ters of parallel composition operators only. As a consequence, scaling the system 
specification to a higher number of components does not make it necessary to 
change the internal behavior of processes. For instance, in 0 we have scaled 
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the system of the router specification to four possible channels for each destina- 
tion, by simply adding several instances of Routing and Channel terms, and by 
appropriately adjusting the parameters of parallel composition operators. 



5.2 Performance Analysis 

In order to derive performance measures from the multi-path router specifica- 
tion, we resorted to the software tool TwoTowers P], that has been recently 
extended to support the generative-reactive approach presented in this paper. 
Such a tool also implements an algebraic reward based method to specify and 
derive performance measures. The results of our performance analysis are shown 
in Fig. I I l.’ll In particular, we concentrated on two main metrics. On the one 
hand, we evaluate the throughput of the system at steady state, represented 
by occurrences of actions of type transmai, transma 2 , transmbi, and transmb 2 , 
by attaching a reward equal to 1 to the above actions and a reward equal to 
0 to each other action. Since the throughput is a frequency expressed in terms 
of number of actions executed per time unit and the time unit is 1/2002950 
seconds, we have to multiply the throughput resulting from the analysis of the 
Markov Chain by 2002950 in order to obtain the results (expressed in seconds) 
shown in our tables | 2 ]. On the other hand, we evaluate the router idleness at 
steady state in terms of the percentage of time the IMP is inactive. The router 
is considered to be idle when no packet is currently inside the IMP, i.e. when it 
executes actions of type idle. Therefore we attach a reward equal to 1 to such 
actions and a reward equal to 0 to each other action. Since the time unit of the 
Router process (half a microsecond) is scaled by a factor (1 — pi) -p 2 ~ 0.998527 
and the reward gained by actions is related to the duration of the corresponding 
activity expressed in half microseconds, due to the time unit change we must 
multiply each reward by 1/0.998527 before analyzing the Markov Chain P]. 

For each conducted analysis, we assumed that the incoming traffic for the 
destinations a and b follows the same Bernoulli distribution of parameter r = 
ra = rb. The figures show how the performance measures change when we vary 
r from 0.1 (sometimes 0.01) to 0.9. In this way we observe the system behavior 
under various levels of workload ranging from 10% (or 1%) to 90%. 

We start by evaluating the system throughput under different circumstances. 
We first consider the situation in which p = \, i.e. the packets destined to a and b 
are managed at the same speed by the Switch process, and parameters qa and qb 
reflect the bandwidth distribution over channels directed to destinations a and 
6 , respectively. Since channel ai can deliver 100 packets per second and channel 
02 can deliver 400 packets per second, we take qa = 100/(100 -I- 400) = 1/5 
(the ratio of the bandwidth of channel oi over the overall bandwidth of the 
channels directed to a) , so that packets are probabilistically distributed between 
channels oi and 02 in the optimal way. Similarly, since channel b\ can deliver 
200 packets per second and channel 62 can deliver 250 packets per second, we 
take qb = 200/(200 -I- 250) = 4/9. The obtained results are reported in Fig. Q 
As we can see in Fig. HB) the curve representing the total system throughput 
is characterized by a high slope in correspondence of a low workload and a 
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(A) Router Idleness 



(B) Throughput (p=1/2, qa=1/5, qb=4/9) 





Load factor ra = rb 



Load factor ra = rb 



(C) Throughput (p=1/2, qa=1/5) 



(D) Throughput (p=1/2, qb=4/9) 





Load tactor ra 



Load tactor rb 



Fig. 1. Idleness and Throughput of the Multi-path Router 

quite flat slope when the load factor increases over the 50%. This because, for 
packets with a given destination, the bandwidth associated with the channels 
directed to that destination is about one half of the maximum bandwidth of the 
incoming traffic. Simply put, when the parameter r of the Bernoulli distribution 
representing the incoming traffic reaches the 50%, the outgoing channel is almost 
fully occupied, hence a further increment of r gives rise to a very small increment 
of the outgoing throughput. Another expected result is that the throughput of 
packets destined to a is slightly greater than the throughput of packets destined 
to b. This because the overall bandwidth of the channels directed to a is 500 
packets per second, while it is 450 packets per second for the channels directed 
to b. Fig. mc) and Dd) report the throughput for each single channel oi, Q 2 , 
bi and & 2 - In the case of a± and 02 the distance between the two curves is quite 
great, this because a± has just one fourth of the bandwidth of 02 . As expected 
such a difference is smaller in the case of bi and 62 , because their bandwidth is 
quite similar (200 and 250 packets per second, respectively). 

Since in a realistic framework the value of parameters qa and qb are estab- 
lished by the multi-path routing algorithm governing the IMP according to the 
network conditions (e.g. estimated time for a packet to reach a destination via a 
particular path), we study the effect on the throughput of adopting parameters 
qa and qb which do not reflect the bandwidth distribution over the outgoing 
channels. The results of such an analysis are reported in Fig. 0 In particular. 
Fig. 0(A) shows how the throughput of ai and 02 varies when changing the value 
of qa from | to |, i.e. by exchanging the value of qa and 1 — qa. For the sake 
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(A) Throughput (p=1/2) 




Fig. 2. Throughput obtained 



(B) Throughput (p=1/2) 




varying parameters qa and qb 



of clarity we report the curves obtained for both qa = ^ and qa = ^. We can 
observe that the parameter qa does not play a significant role when the router 
is congested. This because under a heavy workload both routing processes are 
hardly occupied and in most cases at least one of them is busy. In such a situa- 
tion the parameter qa is often not used, because when a routing process is busy 
an arriving packet destined to a is passed to the other routing process with prob- 
ability 1. As a consequence the curves of the throughput converge to the same 
values when the load factor ra gets over 50%, i.e. when almost all the bandwidth 
of each channel is exploited. On the other hand, when the incoming workload is 
low, the parameter qa becomes important as it probabilistically decides which 
routing process will deliver the packet, hence increasing the throughput of a rout- 
ing process with respect to the other one (see the quite evident difference among 
the curves when ra gets under the 30%). Fig.|3(B) shows how the throughput of 
the two channels destined to b varies when exchanging the value of qb and 1 — qb. 
With respect to the channels destined to a, in this case the difference between 
the old value of qb (|) and its new value (|) is smaller. This is reflected on the 
results presented in Fig. 0 where the curves for the old and the new value of qb 
are almost overlapped for each value of rb. 

Now we show the role played by parameter p on the system throughput. In or- 
der to merely concentrate on the effects of varying parameter p, we just consider 
the situation in which parameters qa and qb reflect the bandwidth distribution 
over channels directed to destinations a and b, respectively. Parameter p can be 
chosen in order to favour the internal computations of the IMP dedicated to 
packets destined to a (b) with respect to those dedicated to packets destined to 
b (a). To this aim, in Fig. |3we report the throughput of the multi-path router in 
the case p = 0.999, hence when packets destined to b are managed by the IMP 
much more slowly than packets destined to a. As a consequence of the unfair 
behavior of the router, we have that the IMP delivers the packets destined to a 
at the usual speed (the curve for packets destined to a in Fig. ETA) is the same as 
that in Fig. iHB)), but it delays the packets destined to b, hence compromising 
the throughput of such packets. Therefore with respect to the case p = 0.5 the 
overall system throughput decreases (for easy of comparison in Fig. OJA) we also 
report the curve obtained in the case p = 0.5). The comparison with the case 
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the outgoing channels directed to destination b. 

As far as the idleness of the router is concerned, we simply consider the situa- 
tion in which p = ^ and parameters qa and qb reflect the bandwidth distribution 
over channels directed to destinations a and b, respectively. The curve presented 
in Fig. 0A) shows the relation among the inactivity of the router and the load 
factor for the incoming traffic. As expected, the router is almost always idle if 
the workload is low, but the duration of its inactivity phases rapidly converges 
to zero for a load factor greater than 40%. 
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Abstract. The paper introduces a new approach to define process 
algebras with quantified transitions. A mathematical model is in- 
troduced which allows the definition of various classes of process 
algebras including the well known models of untimed, probabilistic and 
stochastic process algebras. For this general mathematical model a 
bisimulation equivalence is dehned and it is shown that the equivalence 
is a congruence according to the operations of the algebra. By means 
of some examples it is shown that the proposed approach allows the 
definition of new classes of process algebras like process algebras over 
the max/plus or min/plus semirings. 

Keywords: process algebras, semiring, bisimulation, congruence. 



1 Introduction 

Process algebras are one of the most important formal specification techniques 
to describe the dynamic behavior of discrete event systems. A large variety of 
different process algebras exists. Process algebras differ in various aspects, e.g., 
in the way how synchronization is defined, in the used equivalence relations and, 
may be most important, in the way how dynamics are quantified. First process 
algebras like Milners CCS P2] or Hoares CSP describe only the functional 
behavior of systems. Thus, the process algebra specifies which action sequences 
can potentially occur, but it does not quantify these sequences or relate a nota- 
tion of time to it. Consequently, process algebras have been extended by relating 
probabilities to transitions png, by assuming a stochastic timing and assign- 
ing rates to transitions and by combining probabilities and time in 

one model C31 Also for probabilistic and stochastic process algebras different 
concrete realizations have been defined which differ in several details. However, 
almost all process algebras have the following features in common. They have 
a well defined formal syntax and a semantics defined as some form of a la- 
beled transition system. The specification is compositional. Finally, equivalence 
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relations exist which allow to compare different specifications and certain equiv- 
alences are congruences according to the operations of the algebra. In summary, 
these properties make this model type successful. 

Known process algebras describe either labeled transition systems or stochas- 
tic processes with transition labels. In the latter case mainly Markov processes 
or Markov chains are considered. However, recently other algebraic models be- 
came very popular for the analysis of discrete event systems, namely min/plus, 
max/plus or min/max systems ^ with applications in the area of communica- 
tion protocols PPITj . queueing networks ^ or real time systems |2|. Instead of 
defining additional specific process algebras for these models, it is more challeng- 
ing to ask for a common framework which has such process algebras a special 
instantiations. In this paper we follow the latter approach. We first have to find 
a general representation of the required operations which can be interpreted ade- 
quately in different concrete realizations. It will be shown that the mathematical 
structure for these operations is a semiring. This structure is very general and 
contains the mathematical operations available in different process algebras and 
also in systems like max/plus algebra. Based on the semiring definition a general 
process algebra can be defined by including the common operations available in 
different process algebras like prefix, sum and composition. The underlying se- 
mantics is a labeled transition system where labels consist of actions and an 
additional quantification by elements of the semiring. The interesting point is 
that the mild requirements of a semiring as an algebraic structure are sufficient 
to define a bisimulation in this framework, which appears natural and which is a 
congruence according to the operations of the algebra. This bisimulation is valid 
independently of the semiring one selects for a concrete application. 

The outline of the paper is as follows. In the next section we introduce some 
basic definitions. Afterwards, in Sect.0we present the syntax and an operational 
semantics of Generalized Process Algebra (GPA). Then a bisimulation is defined 
and it is proved that this bisimulation is a congruence according to the operations 
of the algebra. In Sect. 0 we present two concrete realizations of GPA and define 
implicitly process algebras for max/plus and min/plus by means of two examples. 
The paper ends with the conclusions which outline subjects for future research. 

2 Basic Definitions 

Before we present our process algebra, the underlying basic structures are de- 
fined. We start with semirings which are later used to define quantitative values 
of labels. 

Definition 1. A semiring (K, -|- , T ^ 0 ^ 1) js a set K with binary operations 
+ and T defined on K such that the following axioms are satisfied: 

+ , ^ are associative, and + is commutative, right and left distributive laws 
hold for + and , O and II are the additive and multiplicative identities with 
O yf 1, and for allkGKk'i'0 = 0'i'k = 0 holds. 



To make the notation simpler we use sometimes K for the whole semiring. 
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Some typical examples for semirings are (B, V, A, 0, 1) the Boolean semiring, 
(R, +,-,0,1) the semiring over the real numbers with the usual addition and 
multiplication, (RU{oo},min, +, oo,0) the min/+ semiring, (RU{— oo},max, +, 
oo,0) the max/+ semiring or (R.U {oo} U {—oo}, max, min, oo,0) the max/min 
semiring. In the sequel we use for the addition + and for multiplication or 
no operator symbol as usual in most semirings. 

Semiring operations can be easily extended to define operations on vectors 
and matrices. Let p S K" be a n-dimensional vector and Q G K"’” a n x n 
square matrix both including elements from semiring K, then q = pQ equals a 
vector in K" which is defined as 

1 

- — ~n—l ^ ^ 

where X)i=o ■ + On-i- Dot product of vectors, matrix addition and 

multiplication can be defined similarly. 

Now we can define transition systems where transitions are labeled with 
symbols from a finite alphabet and from a semiring. This notion is subsequently 
used to define the semantics of our process algebra. 

Definition 2. A (finite) multi labeled transition system (MLTS) is a b tuple 
MLTS = (5, ^ct, K, T, mi), where S is the state space which is countable (fi- 
nite), Act is a finite set of transition labels, K is a semiring used for the definition 
of transition costs, T : S x Act x 5 — ^ K is the transition function, ini : 5 — >■ K 
is the initialization function. 

An MLTS can be interpreted as a graph where each transition is labeled by a 
symbol to define the operation and a cost to perform the transition. The meaning 
of costs can be very different depending on the semiring one considers. If the state 
space is finite, the MLTS corresponds to an automaton with transition costs 
or weighted automata, a well known model in automata theory. Each MLTS 
can be characterized by a set of matrices, one for each a G Act and a vector. 
Let Qa be the matrix for label a G Act, then Qa{x,y) = T{x,a,y). p° is the 
initial vector of the system such that p*^(a;) = ini{x). Function ini gives an initial 
quantification for all states, which depends on the concrete application, e.g. to 
consider reachability from a certain state Si G 5 a common definition of ini is 
ini{s) = II if s = Si and O otherwise. The set of initial states Sini contains all 
s G 5 with ini{s) yf O. 

For many, but not for all, applications, the dynamics of the system can be 
described by vector matrix products. We briefly describe this kind of dynamics 
here. Usually one can distinguish between generative and reactive models |2f)j . In 
the reactive case, the system reacts to the behavior of some environment which 
determines the label that occurs next. A state of the system is described by a 
vector with elements from K. p'^ is the initial state vector and if p* describes 
the current state and a G Act is the label chosen by the environment, then 
pfe+i _ describes the next state. In a generative situation, the system can 

decide by itself which transition occurs next. In this situation the next state is 
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computed from = P^J2aeAct^a.- In both cases we may analyze the system 
according to the state reached after a transition sequence. Thus, let G Act*, 
u>i be the i-th symbol in oj and |w| be the length of uj. p“ is the state reached 
after observing u> starting with p°. The vector p“ can be computed as 

|lxl| 



n— 1 ^ ^ 

where rii=o = oq ^ ^ an-i- We may go further in analyzing the behavior 

of a system by analyzing its generation of actions (traces) or its reaction to the 
input from the environment. Define ^ transition 

sequence lo. Observe that costs can have different meanings in each concrete 
realization. 

Examples: If we use (B, V, A, 0, 1) in the above definition and ini assigns 1 to 
the initial state and 0 to all other states, then the resulting model describes the 
well known labeled transition system. Vector p^ describes the set of reachable 
states after k transitions. Thus, p^(a;) = 1 implies that x is reachable after k 
transitions and in the reactive case, the observed labels have been defined by the 
environment. For deterministic systems, each matrix row contains one element 
equal to 1. 

In a similar way, probabilistic systems can be defined. For probabilistic sys- 
tems, p*' has to define a probability distribution. Depending on the used seman- 
tics, matrices Qa are stochastic or substochastic matrices. In the reactive case, 
matrices Qa are stochastic, since the environment selects label a. In the gen- 
erative case, each Qa is a substochastic matrix and J2aeAct ^ stochastic 
matrix. 



For stochastic systems describing continuous time Markov chains (CTMCs) 
with transition labels [8I15I20| the above method of computing the dynamic be- 
havior stepwise is not directly applicable. However, by applying the well known 
randomization technique the CTMC with transition labels can be trans- 
formed into a discrete time Markov chain (DTMC) with transition labels and a 



Poisson process. For the DTMC with transition labels vectors p*^ can be com- 
puted and the Poisson process determines the number of transitions which occur 
in any finite interval. 

Another, less common example is the use of max/-|- as semiring and interpret 
the transition labels as reward. For this model element p^(x) describes the max- 
imal reward gained when reaching state x after k steps starting in some state y 
with reward p*^(?/). In the reactive case, the labels of the path from y to x are 
given by the environment. 

For the min/-|- semiring and interpretation of transition labels as costs, we 
have a very similar interpretation. Now p^(x) contains the minimal costs of 
reaching state x after k transitions. 
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3 A General Process Algebra with Transitions Costs 

We now define a process algebra denoted as General Process Algebra (GPA) 
which uses the above concepts. First the syntax of the process algebra is defined, 
afterwards the semantics is introduced. The syntax definition follows the usual 
concepts used in process algebras I2SEI1- 

Definition 3. The set C of terms in Generalized Process Algebra ( GPA ) over 
a set of finite transition labels Act including label t and a semiring K is defined 
by V ::= 0 | (a, k).V \V + V\ V\\sV \ V\L \ V = V where a e Act, fc e K 
and S,L G Act \ {r}. 

To avoid too many parenthesis we define among the operations the following 
decreasing priorities: Hiding, Prefix, Gomposition and Summation. We also allow 
for recursion by defining equations, i.e. we use constants as abbreviations by 
defining A = P which means that A is an abbreviation for P. In contrast to = 
we use A = B to denote that A and B are syntactically identical. A term in the 
algebra is denoted as an agent. 

The intuitive meaning of the operators is as follows: 

— 0 : describes the termination of an agent which afterwards cannot perform 
any actions. 

— (a,k).A : action a is performed and afterwards the agent behaves like A, the 
cost of performing action a equal k. 

— A + B : the agent behaves either like A or like B. 

— A \\s B : A and B proceed concurrently and independently on all actions 
which are not in S. All actions from S have to be performed as joint actions 
by both agents. 

— A \ L : actions from the set L are hidden, i.e., they become r actions which 
are no longer usable in joint actions with an environment. 

— B=A : describes that agent B behaves like agent A. 

The semantics of an agent A G £ is a MLTS. As usual in process algebras, we 
cannot distinguish between an agent and a state. An agent and all its derivatives 

form the state space of a system. We use the notation A B for T{A, a,B) = k 
if fc O. Semantic rules are given in an operational style of the form 

premisei . . . premiscn , \ , \ 

; (name) [condition) 

conclusion 

to be read as: 

If condition is satisfied, the rule (name) can be applied and it can be deduced 
that conclusion holds in case of the assumptions premisei ■ ■ ■ premisen hold. 

In all semantic rules we define, conditions are introduced such that transitions 
with cost O are not explicitly generated. However, as usual in labeled transitions 
systems, if no transition between two states exists, then there is implicitly a 
transition with cost O. Observe that the meaning of O in all semirings is such 
that 0^0 = 0 which means that the contribution of a zero element to a vector 
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matrix product is zero. For the prefix and choice operators we have the following 
operational semantics. 



(a,/c) . A — AA 



(pr) (fc yf I 



A.-fh^A' 



rA' 



(+) 



^jT{Ai, a, A') and I is some index set. 



J 

ks 



e I, 



Note that the choice 



where ks = 

operator can yield the situation of multiple arcs with same labels between agents. 
The above rule ensures that these arcs result as one whose transition function 
takes a single, well-defined value of k^. For parallel composition, the following 
four semantic rules are defined. 



A^A' 



A'^A 

Ia'IIsB 



I Si 



%-A or B — 



3) 



^lIsS 



A\\sB 



a ^ S, 
k^O 
a ^ S, 
k -\- I ^ ( 



B^B' 
AllsB — 
^A' 



B'^B 
■A\\sB' 
B^B' 



IS2 



J 



A\\sB 



a ^ S, 
k yf I 

a G S, 
k'^l yf C 



Rules llsj, 1 1 52, and H53 describe how both agents proceed independently on 
actions a ^ S, rule Hsg considers the special case of independent self loops with 
the same label. The latter is necessary to avoid a reduction of transitions and 
to retain a well-defined transition function if a parallel composition without 
synchronisation applies. If only one condition of the premise in H53 is satisfied, 

say A A', then let I = O for the definition of the conclusion (and vice 
versa). Actions a G S are performed as joined actions. The costs of the joint 
transitions equals the product T of the costs of both transitions (rule jjs^) with 
respect to the semiring. The definition of costs of synchronized transitions is one 
of the most discussed questions during the development of stochastic process 
algebras. Several approaches have been proposed in the literature m and there 
are arguments for and against any of these solutions. We do not claim that 
the use of T is always the best way to define the costs. However, it is a natural 
solution from a mathematical viewpoint and we found reasonable interpretations 
for this choice in all semirings we considered so far. 

The semantics of the hiding operator is defined as 

(\i) (a ^ L, fc yf O) and 



A\L 



z,k 



%^A' 



A\LAAa'\L 



A'\L 

^(\2) 



({ai> 



J C LU{T},ks O ), 



where k^ = Transitions which are not hidden occur as before and 

hidden transitions between two states appear as a single transition with label r 
where costs are accumulated. Note that a transition with label r can be part of 
the premises as well. 

All operations we have considered so far, allow only the specification of finite 
behaviors. To define an agent with an infinite behavior one has to define A = B 
where A occurs in B. E.g., A = {a,k).A is an equation describing an infinite 
behavior. The semantics of a constant is then given by 



A^ A' 

B^A' 



(=) {B = A). 
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We sometimes use also variables for processes which can be bound to agents. 
We use the notation A{BjX^ when every occurrence of process variable X in 
A is replaced by agent B. We denote a variable as free if it is not bound to an 
agent. Following Milner a term of GPA is denoted as an agent if it contains no 
free variables. 

What we have defined in this section is rather typical for a process alge- 
bra with an operational semantics. The only difference is that we allow general 
semirings to define the quantitative values. Concrete applications of this concept 
are presented in Section 0 The semantics of an agent A is an MLTS where the 
state space Sa contains A and all its derivatives reachable by applying the se- 
mantic rules. The initialization function ini assigns t to B G Sa if B=A and O 
otherwise. In this way agent A generates an MLTS. 

Define V[A\ as the set of all successors of agent A, i.e., V[A\ = {B G C\3a G 

Act : A B}. Similarly Pa [A] is the set of all successors of A which are 
reachable by one a labeled transition and P*[A] is the transitive and reflexive 
closure of relation P[A] and equals Sa- 



4 Bisimulation for GPA 



Two of the major advantages of process algebras are compositionality and the 
availability of equivalences which are congruences according to the operations 
of the algebra. One of the most famous equivalence relations is bisimulation 
which is well known for untimed systems IJ IL.il and has been subsequently 
extended to probabilistic 122! as well as to stochastic systems !7tf|l(7H9|. In 0 
it has been shown that bisimulation can be defined for automata with transition 
costs where values of costs are elements of a semiring and the behavior of the 
automaton is defined using vector matrix operations derived from the addition 
and multiplication of the semiring. Of course, automata with transition costs are 
identical to finite state MLTS, the semantic model of GPA. We first introduce 
bisimulation for GPA and prove afterwards the congruence property according 
to the operations of GPA. 

The definition of bisimulation requires some additional notations as a pre- 
requisite. We restrict bisimulations to equivalence relations and assume that the 
MLTS is of the finite branching type, i.e., the number of transitions with non- 
zero costs leaving a state is finite. Let TZ be an equivalence relation on £ x £. 
CCn is the set of equivalence classes of TZ, Cn is an equivalence class of TZ and 
Cti\B] is the equivalence class to which agent B belongs. 

Definition 4. An equivalenee relation TZ on Cy. C is a bisimulation if and only 
if for all (A, B) G TZ and all a G Act: 

1. ini{A) = ini{B) ; 

2. tfA^C^ for some C G C, then 



y T{A,a,D) = y 

^£>gCk[C] ^ ^ 



D€Cti[C] 



T{B,a,D) ; 



^ Observe that A C implies k <D which follows from our semantic rules. 
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3. if B C for some C G C, then 



E 



DdC-R.[C] 



T(B,a,D)=V ^ 



d^c-r.[C] 



This definition corresponds to the usual definition of bisimulation for un- 
timed probabilistic 1221 or stochastic jS| systems. Note that in the classical 
bisimulation 1221, one considers existence of specific agents C, C which need to 
be bisimilar which is the same as to ask for elements D G C-ji \C] . Like for other 
bisimulations the following theorem holds. 

Theorem 1. Let TZ\, TZ 2 be two bisimulations on C, then TZ — TZ\ U7^2 is also 
a bisimulation. 

Proof. The proof can be found in Pj and follows the same line of argumentation 
as the proof for the classical theorem in Boolean systems I2B1- □ 

The theorem implies that the largest bisimulation exists as the union of all 
bisimulations. 

Definition 5. Two agents A and B are bisimilar, denoted as A B, if a bisim- 
ulation TZ exists such that {A, B) G TZ. 

The term equivalence implies that equivalent agents behave identically in 
some sense. This is also the case for a bisimilar agent in our general setting. An 
example is given in the following theorem where we come back to the notion 
of transition cost described in Section |5| with c^, = X^sgsP'^('®) ^ 
transition sequence w. 

Theorem 2. Let A and B be agents generating MLTSa and MLTSb with 
Act = Act A = Acts. If A ^ B, then for each ca G Act*, c{f = c^ where c^,c^ 
are the costs for performing to in A and B, respectively. 



Proof. The proof follows by induction. We first prove that for all Cn G CCn, 

kGn, 



holds. The relation is true because p°(A) = ini{A) = ini(B) = p°(B) holds due 
to A ~ i? and Def. 2]and p°(C) = O for all C ^ AV B according to the definition 
of ini for the MLTS semantics of an agent. Now assume that the relation holds 
for a string to. We prove the induction to a string w; a where a G Act and w; a is 
the concatenation of lo and a. 
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The first equation holds due to the cost assignment by a vector matrix multiplica- 
tion, see . The second equation is based on the requirements for a bisimulation 
(Def.0). The third equation follows from dD like the first one. The identity of 
the vectors implies the identities of the cost functions which can be seen by the 
following equations. 

= ScKGCCTCSc’GCTCnX’*[A]P“(C') ■ 

ScKGCC-RScGCKrTD*[B]P“(^) ’ 



(EBGB*[A]ma,7^)) = 

(Edg'd* [B] ~ 



□ 

The theorem introduces some form of trace equivalence for GPA which is 
observed by bisimilar agents. 

An equivalence relation is especially useful, if it is a congruence according to 
the operations of the algebra. In this case, equivalence assures that equivalent 
agents behave “equivalently” in all possible environments and equivalent agents 
can be substituted which allows the interleaving of composition and aggregation 
miHi. The following theoremsO, EJstate that bisimulation is indeed a congruence 
according to the operations of GPA. 

Theorem 3. Let A, B,C € C with A B , then the following relations hold 

1. (a, k).A ~ (a, k).B for all a G Act and A: G K \ {O}, 

2. A + C B + C, 

3. A \ L B \ L for all L C Act \ {t}, and 

I AlIsC - B\\sC for all S C Act\ {t}. 

The proofs follow from the proofs for untimed process algebras or stochastic 
process algebras |H|. Note that we have a strong similarity to untimed process 
algebras if one focuses only on labels a G Act, such that the crucial point for 
bisimulation is in the treatment of elements of the semiring. A key observation is 
that the semantics of Prefix, Ghoice and Hiding preserve existence of transitions 
and that if multiple transitions between two states are merged into a single 
one, elements of the semiring are added by + . This matches requirements for 
the bisimulation which adds elements of the semiring by -I- as well. Due to 
associativity and commutativity of -I- in Def. ^the order in which elements are 
added does not matter. We omit proofs of prefix, choice and hiding for Theorem 
0 since these are lengthy and do not give further insight. We focus on the proof 
of composition, since this one is the most interesting, especially rule ||s 4 , where 
elements of the semiring are multiplied. Observe that the distributive laws of 
the semiring allow to conclude that for finite index sets J, J, and K and Oi, bj, 
Ck elements of the semiring for alliGl,jGJ,kGK holds 

“ (EiG/®*) ■ (EfcGiC'*fc) “ (EjGJ^i) ’ (EfcGiC'*^) 
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if Having this in mind, the argumentation on equations for 

proving rule ||s 4 are easier to follow. The necessity of associativity and distribu- 
tivity of over + for existence of congruences is well known, e.g. |1 Sj . The proof 
follows the same line of argumentation as Milner’s proof of strong bisimulation 
in [23j . 

Proof, of composition in Theorem El 

Let TZ be an equivalence relation such that (AUsC, H||sC) G TZif and only if A ^ 
B. We have to prove that 7^ is a bisimulation. Since ini{A\\sC) = ini{B\\sC) = 1 
and ini{D) = O for D ^ (AUsC) V (HUsC) by definition of the MLTS generated 
by HjlsC, resp. B\\sC, the first condition of a bisimulation is satisfied. 

Suppose (AlIsC, 5115(7) S TZ, let H||5(7 E then there are four cases 
according to the four semantic rules of composition; cases 1-3 consider a ^ S: 

Case 1: A-^ A', A' and E = A'\\sC 

We use the notation D G C^[A'] as in Def. E| with ^ as symbol TZ there. Since 
A ^ B we have O ^ [A'] T{A,a,D) — X)dgc...[A'] T{B,a,D) so we have 

D G C^[A'] where B\\sC D\\sC and (A'||5(7, 5||5(7) G TZ. 

Case 2: C ^ C , C jk C and E = A\\sC 

Then also B\\sC ^ B\\sC’ and (A|ls(7', 5||5(7') G TZ. 

Case 3: A A or C ^ C and E = H||s(7, k = I + m. 

If only one of the two conditions holds, then we have the same argumentation as 
for case 1 or 2, respectively. If both agents contain a self loop with label a, then 

from A ~ 5 we have O yf J2oeC = J2dgC 

D G C^[A] we have B D, but since A G C^[A] and A ^ B, also B G C^[A], 
such that (A||5(7, 5||5(7) G TZ and 

Er5GC.[A]7’((^llsC),a,(5||5C)) + E^’GC.[C]n(^llsC),a,(A||5F)) = 
EDec^[B]Tm\sC),a, (5||5(7)) + EFec^ic]Tm\sC),a, (HIU^)) 

due to the associative and commutative law for + in the semiring. 

Case 4: a G S, A ^ A' , C ^ C and E = A'\\sC, k = l^m. 

By A ^ B we have B B' with B' G so clearly B\\sC B'\\sC 

and {A'WsC , B'\\sC) G TZ. It remains to show equality of transition costs in the 
semiring. For each Cn G CCn we have: 

%D\\sF)ec^nmsC),^ {D\\sF)) = T{C, a^) ^ %^^ccXoec^nA, a, D) 
= T{C, a, F)-Y.c^^ccj:DecT{B, a, D) = T.(D\\sF)ecJ^m\sC),a, {D\\sF)) 



due to distributive laws for + and in the semiring. This shows that the 
identity of transition costs holds. By a symmetric argument we complete the 
proof. □ 
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It remains to show that bisimulation is a congruence for recursive behaviors. 
We first define under which conditions agents including variables are bisimilar. 

Definition 6. Let A,B £ C and let A, B include variables Xi (i £ I for index 
set I) at most. A ^ B if and only if for all agents Ci (i G I ) 



A{CJX,} - B{C,/X,} . 



Theorem 4. Let C,D G C, C ^ D and let C and D both contain variable X. 
Let A = C{A/X'\ and B = {B/ X}, then A B. 

Proof. Since recursion by defining equations does not involve elements of the 
semiring, the proof follows from the proof for the Boolean model, e.g. as given 

in jZg. □ 

By means of the above theorems equivalence transformations can be defined 
at the syntactical level similar to the rules in process algebras like CCS m- 

5 Examples and Realizations 

It is straightforward to define different process algebras using the general concept 
which has been presented in the previous sections. We give three application 
examples in the following subsections. 

5.1 A Probabilistic Process Algebra 

Different realizations of probabilistic processes exist in the literature [2SE2|. 
On particular problem with probabilistic processes occurs when composition 
is introduced because after composition the sums of probabilities of outgoing 
transitions for an agent might be smaller or larger than 1. Some calculi introduce 
rescaling of probabilities in such a case which, however, may yield some problems 
with an appropriate definition of bisimulation as shown in m- We avoid this 
problem by imposing sufficient restrictions on our definition of a probabilistic 
process algebra for the semiring (K+, +, •, 0, 1). For each agent A and each 
a G Act the following condition holds. 

^ T(A,a,B)G{0,l} (2) 

BeValA] 

Thus an agent can either perform action a or it cannot perform this action. If 
an action is possible, then the successor agent might be chosen probabilistically. 
In the notation of this describes a reactive system. We assume in the sequel 
that Act does not contain label r and all labels can be used for synchronization. 
Note that hiding makes it difficult to retain property 021). 

Corollary 1. Let A and B be two agents observing ^ and let AcIa and Acts 
not contain label r, then the following agents also observe 
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— (a, 1).A + (6, 1).B with a ^ b, 

- (a,p).A + (a, 1 — p).B for 0 < p < I, 

- A + B if Act A n ActB = 0, 

— if {Act A n Acts) \ S' = 0 

and any combination of the above agents. 

Reactive systems are usually driven by some scheduler a component which 
resolves the nondeterminism by choosing actions to perform. An agent AS is a 
scheduler if for all A G T>*[AS] the following condition holds. 

E E T{A,a,B)G {0,1} (3) 

aGAci\{r} B^T)[A] 

An agent A G T>*[AS] is terminating if T>[A] = 0. Each composition AS\\ActB 
where AS observes m and B observes (0 is stochastic process where the sum of 
outgoing probabilities is 1.0 or 0.0 for each agent. The resulting process reaches 
a terminating state if the scheduler reaches a terminating state and it reaches a 
deadlock state if it reaches a state without successors but the scheduler is not in 
a terminating state. Alternatively, one might be interested in an infinite behavior 
which means that the system never reaches a state without successors. 

As a simple example we consider the well known dining philosophers problem 
where philosophers pick up one fork after the other, but a philosopher may first 
pick up the left or the right fork depending on their availability. We consider 
a system with N philosophers and forks where philosopher n and fork n are 
described by the following terms in GPA. 

PHf = {gl, l).(g;, l).PHf + (g^, l).(gi, l).PHl 
PHI = {pi, l).{pi, l).PHl + {pi, l).{pi, l).PHl 

= {gi, l).{pi, 1).F„ + 1).(pL+i, l)-^n 

where n + 1 in g„+i and Pn+i is performed modulo N to have a cyclic system. 
The overall system results from composition of philosophers PH{{, forks Fn and a 
scheduler AC with S = Two possible schedulers which allow 

an infinite behavior of the system are the following two. 

ACi = {g{, 1/N).{g{, l).{p[, l).{p\, l).AC, + . . . 

+ {g{„ l/iV).«, 1 ).(p5v, 

AC 2 = {g[,l).{g\, l).{p[, l).{pl, 1) ^)-(Pn, ^)-AC 2 

Of course, other schedulers allowing more parallelism can be defined as well. 

5.2 Max/Plus Process Algebra 

As a second example we consider GPA over the semiring (K.U {— oo}, max, +, 
— oo,0). To the best of our knowledge no process algebra has been proposed 
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for this semiring yet. However, this model is well suited for the formulation of 
deterministic scheduling problems or several other optimization problems. 

Deterministic scheduling problems are often formulated using marked graphs, 
a specific class of Petri nets without choices. As an alternative formalism one may 
as well describe these systems using GPA with the advantage of compositionality 
and the availability of a well defined equivalence. For the application of max/+ 
algebra to the analysis of timed marked graphs we refer to 0 • 

Max/+ algebra can only be applied for the analysis of Petri nets without 
choices. This restriction is, of course, taken over to GPA. Thus, an agent for the 
analysis of a scheduling problem must not include choices which means that the 
+ operation cannot be used for the specification of agents. 

Since the formulation of models in GPA over the max/+ semiring is straight- 
forward, we consider a simple example. The example consists of two sources 
which generate parts of raw material to be assembled by a machine. The two 
sources are described by agent A and B and generate their material after con- 
stant times tA and ts- 

A = {T,tA)-Ai Ai = {a,tp)-A B = Bi = {a,tp)-B 

The parts are assembled by a machine described by component C. The machine 
picks up the parts if both are available by a transition with label a where picking 
takes time tp for each part. It subsequently assembles both parts and offers the 
assembled part to some environment via a transition with label b. We assume 
that the machine has no intermediate buffer such that the assembled part has 
to be first delivered before processing of a new part can start. 

C'=(a,0).Ci Ci = (T,tc)-C2 C2 = {b,0).C 

The whole system is composed of the three components. Label a is hidden be- 
cause the environment interacts with the system only via b. 

Sys = {{A\\aB)\UC)\{a} 

The set 'D*[Sys\ contains the following 12 states which are denoted by the state 
of the components. 

1){A,B,C) 2){A,B,C,) 3){A,B,C2) A) (A,B^,C) 

5){A,Bi,Ci) 6 ){A,Bi,C2) 7){AuB,C) 8){Ai,B,Ci) 

9) (Ai,B,C2) 10) (Ai,Hi,C) 11) (Ai,Hi,Ci) 12) (Ai,Hi,C2) 

The MLTS of the system is shown in Fig. d In the picture, transition la- 
bels are not shown instead r-labeled transitions are denoted by solid arcs, 
whereas 6-labeled transitions are described by dashed arcs. Transition costs 
are written near the arcs. Observe that O = — oo in this semiring such that 
all arcs which are not shown have weight — oo. li t a = is, then the relation 
TZ — {(1), (2), (3), (4, 7), (5, 8), (6, 9), (10), (11), (12)} is a bisimulation for the 
system and a smaller equivalent system can be generated which contains 9 in- 
stead of 12 states. Equivalent agents are surrounded by dotted boxes in the 
figure. The system can be analyzed according to the completion time of parts 
by computing vector matrix products in the general form described above. 
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Fig. 1. MLTS of the scheduling example. 



5.3 Min/Plus Process Algebra 

Max/+ algebra is often used for systems with synchronization, where weights 
describe delays such that at a synchronization point one agent has to wait for 
another to do the synchronization. In min/+ algebra, + is becomes minimum. 
This is useful to formulate optimization problems like shortest path problems or 
Markov decision problems. Of course, Min/+ and Max/+ are very similar since 
one can substitute the other by simply using negative values. 

As an example for min/+ systems we consider a model for the computation 
of minimum traveling costs. The scenario is simple: A traveler wants to go from 
a source place to a destination place. Between source and destination several 
cities are located and the traveler can choose different ways and different means 
of transport. We use label b for buses, t for trains and c for a cab and consider 
as an example a route with 6 cities. City 1 is the source, city 7 is the destination 
which is denoted as agent 0 in our specification. The following agent defines the 
connections between cities. 

Co = (6,2).C2 + (6, 2).C4 + (c, 6).Cs Ci = (6, l).Co + (f,2).Cs 

C2 = {t, 2). Cl + (c, sj.Co C3 = (c, 5).0 + (t, 1).C4 + (c, 4).Cs 

C4 = (t, 4).0 + (c, 3).C5 C5 = (6, 2).0 + (6, 2) .C3 + (t, 1).C4 

A traveler can now be defined to synchronize with agent Cq. Consider first a 
traveler who can use arbitrary means of transport. 



ri = (6,0).Ti + (c,0).Ti + (t,0).Ti 

The agent Sys\ = (Co||6,c,t7i) \ {b,c,t} can be used to determine the minimum 
traveling costs. The MLTS for this system contains 7 states. Initially Co\\b^c,tT 
receives cost 1 = 0 and all remaining states receive costs O = 00. Costs can 
be computed by computing vectors p^; if p* = p^“''^, then p^(A) contains the 
minimal costs of reaching agent A G 'D*[Sysi] from the initial state. The used 
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algorithm is, of course, the well known Bellman Ford algorithm for shortest path 
problems Since we are interested in the costs of reaching the destination, 
p^(0) is of interest. In this example we obtain minimal costs of 6. 

We may as well define other travelers. A traveler who is not allowed to take 
a taxi is defined as T2 and a traveler who wants to move at most once by train 
or bus is defined as T3. 

T2 = (6,0).Ti + (t,0).T2 T3 = (c,0).T3 + (6,0).T3i + (t,0).T3i T31 = (c,0).T3i 

Both agents, T2 and T3 can be composed with Cq like T\. For T2 we obtain the 
same minimum as for T\ , wheras T3 has costs of 7 which is the additional price 
to use a cab. 

6 Conclusions 

In this paper we proposed a new and generic approach to define process algebras 
with quantified transitions. It has been shown that using an arbitrary semiring 
structure it is possible to define a process algebra with transition costs described 
by the elements of the semiring. Different behaviors of an agent are expressed 
by the operations of the semiring. The general process algebra GPA has been 
introduced based on the presented concepts. A very interesting aspect of this 
general approach is that a bisimulation equivalence can be defined based only 
on the requirements of the semiring and that the equivalence is a congruence 
according to the operations of the algebra. 

It has been shown that by choosing concrete semirings we obtain process 
algebras very similar to existing process algebras. In this way it is easy to define 
untimed, probabilistic or stochastic algebras as it has already been done. We do 
not elaborate on the derivation of a stochastic algebra in our framework. Stochas- 
tic algebras mainly differ in the selection of rates in case of synchronisation HH|. 
Since composition is based on F in GPA, selection of semiring (K+, -k, •, 0, 1) 
yields a stochastic algebra in the flavor of MTIPP m- Rather than deriving ex- 
isting approaches, we use other semirings to achieve new process algebras includ- 
ing an appropriate notation of bisimulation. In this paper we present a max/-|- 
and a min/-|- process algebra which are useful for the formulation of various opti- 
mization problems. The semirings max/-|- and min/-|- have been considered here 
only for constant values. The resulting model is commonly used for the analysis 
of deterministic problems (,'-11 1 1 )j . Additionally, max/-|- algebra is applied for the 
analysis of stochastic systems by considering non-decreasing functions in- 

stead of constants as elements of the semiring. It is obvious that most of the steps 
proposed in this paper can be taken over to this more general model. However, 
some extensions have to be introduced and are considered in future research. 
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Abstract. Many formalisms and solution methods exist for perfor- 
mance and dependability modeling. However, different formalisms have 
different advantages and strengths, and no one formalism is univer- 
sally used. The Mobius tool was built to provide multi-formalism multi- 
solution modeling, and allows the modeler to develop models in any 
supported formalism. A formalism can be implemented in Mobius if a 
mapping can be provided to the Mobius Abstract Functional Interface, 
which includes a notion of state and a notion of how state changes over 
time. We describe a way to map PEPA, a stochastic process algebra, to 
the abstract functional interface. This gives Mobius users the opportunity 
to make use of stochastic process algebra models in their performance 
and dependability models. 



1 Introduction 

Many performance and dependability modeling formalisms and model solution 
methods have been developed. The most suitable formalism for a particular 
model depends on many factors, including the experience of the modeler, the 
results required, and the resources available to solve the model. In addition, 
large modeling projects may be split among several teams, each with different 
modeling backgrounds. For these reasons, it is desirable to provide techniques 
and tools for constructing heterogeneous models that may be composed of sub- 
models of different formalisms, each seamlessly interacting with each other and 
with model solvers. The Mobius project aims to provide such techniques and 
tools. 

The theory of Mobius is designed to support heterogeneous modeling by 
the use of an abstract functional interface (AFI I1I2I L a specification that any 
candidate modeling formalism must implement. By doing so, the formalism en- 
sures that its models may share state and synchronize with other models, sup- 
port the definition of performance variables, and be solved using one of several 
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methods. The first formalism supported by Mobius was stochastic activity net- 
works (SANs PI). This paper gives details of the incorporation of a new mod- 
eling formalism, PEPA which is a stochastic process algebra (SPA). Our 
work means that PEPA models may now be specified, composed with other 
submodels, and solved within the Mobius tool. 

Section 13 provides some background on the Mobius framework and tool, and 
a review of the required SPA concepts. Section Pjdiscusses equivalence-sharing in 
Mobius, and some basic notions of state for a PEPA process. Section 0 presents 
an extension to PEPA that makes use of process parameters, and gives details of 
how this extension can be employed in providing a useful and intuitive mapping 
to the Mobius AFI. Section 0 presents an example of modeling with PEPA 
using the Mobius tool, and finally Section Q discusses ways in which the Mobius 
framework could be extended in the future, and concludes the paper. 



2 Background 

In this section, we first describe the Mobius framework and its implementation as 
the Mobius tool. Following that, we provide a reminder of the relevant features 
of our stochastic process algebra, PEPA. 



2.1 The Mobius Framework 

The Mobius framework ra provides an abstract set of requirements for building 
a particular modeling formalism. It is based upon a theory that is motivated by 
many existing modeling formalisms, and seeks to capture the essential compo- 
nents of models built using these formalisms. Any model that is built according 
to the Mobius framework must present a specified interface to other models and 
solvers within the framework. The implementation of this interface is known as 
the AFI. The AFI includes: 

— a set S of state variables, and 

— a set A of actions. 

As described in [B|, a state variable consists of a type, a value, and a dis- 
tribution over initial values. Its value is typically used to represent the state of 
a component or subcomponent, such as the number of customers at a service 
center in a queuing network. The Mobius framework specifies a rich and struc- 
tured set of variable types T. In particular, the integers and subsets of T are all 
state variable types. In theory this set is of infinite size; in practice we assume 
that it is arbitrarily large, but finite. Of course, this is a reasonable assumption 
when it comes to implementing the Mobius tool. The type of a state variable 
is given by a function type : S ^ T; the value of a state variable is given by a 
function val : S ^ V where S is the set of all state variables, and V G T. Two 
state variables si and S 2 are compatible if and only if type(si) = type{s 2 ), and 
are equal if and only if they are compatible, and val{s\) = val(s 2 ). 
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An action consists of a set of action functions (and in general, some state of 
its own, used when building the stochastic process of the model). An action is 
responsible for changing the values of state variables. Just as state variables pro- 
vide an abstraction of model state, actions are intended to be an abstraction of 
the state-changing methods of existing modeling formalisms. For a given action 
a G A, the action function Enahleda : V — >■ bool determines whether or not a is 
“active” in the current state and capable of changing the state at some point 
in the future if uninterrupted. Delay ^ : F — >■ (R — )> [0, 1]) associates a probabil- 
ity distribution function (PDF) with an action in the current state; this PDF 
describes the time from the point of enabling until completion. Given a state. 
Complete^ : V ^ V specifies the state that will result from the completion of the 
action. Because every constructed Mobius model is intended for performance and 
dependability evaluation, non-probabilistic non-determinism is not directly sup- 
ported in the AFL There are more subtle action functions that capture the effect 
that action interruptions have on delay distributions; for more details, see |0|. 



2.2 The Mobius Tool Pj 

Models are built hierarchically with Mobius. The modeler begins by specifying a 
set of one or more atomic models. For example, one of these atomic models may 
be a SAN, just as would have been supplied to the Mobius tool’s predecessor, Ul- 
traSAN 0 . A composed model is built from a set of atomic (or indeed composed) 
models. The Mobius tool currently features two composed model formalisms, 
both based on the notion of equivalence sharing. In both formalisms, submodels 
are linked together such that compatible state variables may be identified. The 
AFI ensures that the formalism need not know the implementation details of its 
submodels’ state. We refer to a set of composed submodels as partner models. 
A benefit of using these composed model formalisms is that the theory of re- 
duced base model construction [B| is employed to construct a lumped stochastic 
process, which often results in significantly smaller state spaces. An atomic or 
composed model is then combined with a performability variable defined on the 
model to generate a solvable model. The performability variable is a description 
of the particular measure that the modeler wishes to calculate. A solvable model 
may be parameterized on a set of global variables to produce a study which is 
composed of a set of experiments. Current work is focusing on inferring statistics 
from a constrained set of experiments using a design of experiments approach |2|. 
Finally, the modeler must choose a technique for solving the collection of exper- 
iments, and the particular solver to use. The Mobius tool provides a number 
of analytical solvers that use a variety of linear algebra techniques for solving 
for steady-state and transient measures. Alternatively it is possible to employ 
an efficient discrete event simulator, which provides confidence intervals for the 
required performance measures. 
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2.3 Stochastic Process Algebras and PEPA 

In recent years, interest has grown in the use of process-algebra-based methodolo- 
gies for performance modeling and evaluation. PEPA ^ is a well-known stochas- 
tic process algebra. Process algebra notations are based on formal languages, and 
the PEPA language provides a small set of combinators, presented below: 

S ■.■.= {a,r).S I 5 + 5 I As 

P ::= P I^P I PjL I A I 5 

Any process described by 5 is termed a sequential eomponent. A process P 
consists of a model eonfiguration of sequential or model components. A PEPA 
model consists of a set of definitions and a system equation, a distinguished PEPA 
term that can be interpreted as the starting state. The language is deliberately 
parsimonious as this helps keep the theory manageable and reduce the proof 
burden. 

For a detailed description of PEPA’s combinators, see 0 . Prefix is the most 
fundamental combinator; a process (a, r).P may perform activity (a, r), which 
has aetion type a and is exponentially distributed with mean 1/r, and then 
evolve into process P. We use a to denote an arbitrary activity. Process P + Q 
expresses a competitive choice between P and Q in which the enabled activities 
of P and Q compete, and a race condition distinguishes the component into 
which the process evolves. The cooperation P <5 is a process that expresses 
the parallel and synchronizing execution of both P and Q. Both components 
proceed independently on activities whose types are not in the set L. However, 
those activities with types captured by L require the concurrent participation 
of both subprocesses, and this results in an activity with a rate that reflects the 
rate of the slower participant. Finally, P/T is the process that hides activities 
with types in L. These become silent activities with type r. This combinator is 
used to express abstraction. 

Processes may be recursively defined as the least solution to a set of equations 

d0f 

where each is of the form A — P. A classical process algebra combinator miss- 
ing from PEPA is the nullary combinator 0, representing the deadlocked process. 
To date, the focus with PEPA modeling has been on steady-state measures, for 
which 0 has little application. 0 can still be represented in PEPA (for example, 

if p (q-j r).P, Q (/3, s).Q, then P Q is deadlocked). We make use of a 
deadlocked process later in the paper. The operational semantics of PEPA infer 
the transitions of a compound process from the transitions of its subcomponents. 

{oL,r) 

If P >P' then P' is called an {{a, r)-) derivative of P. If P >, then there 

exists some P' such that P' is an (a, r) -derivative of P. The derivative set of a 
PEPA process P is denoted by ds{P), and is the smallest set of components that 
is closed under the transitive closure of the transition relation. This captures all 
“reachable states” from P. Both prefix and choice are termed dynamic combina- 
tors, meaning that the combinators do not persist (in general) over transitions. 
In contrast, cooperation and hiding do persist over transitions, and are termed 
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static. From the transition system, a Markov chain can be produced by essen- 
tially discarding activity labels on arcs, providing a performance model. In order 
to perform a steady-state analysis with a finite state space, it is required that 
the underlying Markov chain be irreducible and ergodic. A necessary (but not 
sufficient) condition for this is that the PEPA process be cyclic. A cyclic PEPA 
process is a process with a structure such that no static combinator is within the 
scope of a dynamic combinator. Since static combinators are never “destroyed” 
by activity transitions, this syntactic condition ensures that the structure of 
the process does not grow unboundedly over time. For more details on PEPA, 
including its well-developed equational theory, see P]. 

3 Equivalence Sharing 

In the Mobius framework, models M\ to Mn exhibit an equivalence sharing 
relationship if there exists a state variable Si from each model such that for 
^ ^ i ^ j ^ n, Si and Sj are compatible, and at all times, val{si) = val{sj). 
Mobius uses equivalence sharing relationships in the construction of Repli- 
cate/Join and Graph composed models. The Mobius tool first uses state iden- 
tification and modification methods provided in the implementation of the AFI 
to link together the appropriate portions of submodel state. Any component of 
Mobius that requests information about the state of the composed model must 
also make use of the AFI, ensuring that the correct data is returned. In this way, 
Mobius presents a uniform view of model state, but allows for internal efficien- 
cies in storing composed model state. Equivalence sharing allows for what can 
be viewed as a form of “two-way communication” between models. By altering 
the value of the shared portion of model state, one submodel can influence the 
behavior of its partners, and similarly, can have its behavior influenced by its 
partners. 



3.1 Representations of State 

PEPA’s operational semantics provide a translation from a collection of algebraic 
expressions into a graph model, which leads to a continuous-time Markov chain. 
Each state of the Markov chain is associated with a particular process expression 
(or an equivalence class of process expressions) . Therefore, the PEPA process can 
be viewed as evolving over time according to transition rules, with the current 
state represented by a particular term of the process algebra. As mentioned 
in Section 12.31 we restrict ourselves to considering cyclic PEPA processes only, 
in order to prevent the structure of the PEPA terms from growing without 
bound over time. Since we consider only cyclic PEPA processes, we can view a 
PEPA process as having a tree structure such that no “static” nodes lie below a 
“dynamic” node. Since the static structure is invariant over activity transitions, 
we consider everything at the level of, and below, a dynamic combinator to be an 
evolving subcomponent. A simple tree representation is presented in Figure Q 
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in which the dotted rectangles highlight the subcomponents. 

(((a,r).P' ^ (a,s).Q')/W) ^ (^ + 5) 

Taken together, the individual states of these subcomponents, along with the 
invariant static structure of the process, are enough to characterize the state of 
the model as a whole. 



/{a} 



{a} 





(a,r) P’ 




(a,s) Q’ 



Fig. 1. Highlighting submodel state for Sys 



A simple vector notation for Sys is ((a, r).P', {a, s).Q' , R + S)gy^. The states 
of each sequential component may be enumerated, leading to a simple first map- 
ping to a set of state variables for the AFI. Concretely, for {Ri , . . . , Rn) p and 
1 < i < n, state variable Si is such that type{si) = N and val(si) = enum(Ri). 
However, after considering the use of these state variables in an equivalence 
sharing relationship, we reject this as a workable mapping for two reasons: 

— There is no obviously meaningful way to interpret the natural number enu- 
meration of a sequential component, and there seems to be no compelling 
way in which a partner model could use this data. 

— If a partner model changed the value of one or more state variables, this 
would effect a “jump” in the structure of the PEPA model. This is too 
uncontrolled. 

Instead, our technique will rely on extending PEPA with process parameters 
and providing these as state variables. We next present PEPAjt, our extension 
to PEPA. 

4 Extending PEPA with Process Parameters 

In this section, we present PEPA^, an extension to PEPA that makes use of 
process parameters. Extending process algebras with parameters is certainly 
not a new idea; for example, see What is novel is our use of this 

extension to implement equivalence sharing between models. We present the 
syntax of PEPA^ next. 
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Definition 1 (Syntax of PEPA^) Let A range over a set C of process con- 
stants, As over a set Cs Q C of sequential process constants, and x over a set 
X of process parameters. Let e represent the syntax of arithmetic expressions 
over X, and b represent the syntax of predicates over X. Then the syntax of 
PEPAk is given by: 

S::={a,r).S \ {a\e^r).S \ {a?x,r).S | S' -I- 5 | if 6 then S | As \ As\(f\ 

P ::= P tXip I p/p I A I A[e] I S 

PEPAfc provides the following additions to PEPA: 

Formal Parameters: process variables now have an arity and may be instanti- 
ated with parameters. Furthermore, processes may be specified through the 
definition of equations of the form P[xi , . . . , a;„] Q. 

Guards: process expressions may now be guarded, meaning that the behaviour 
specified by the process expression is only present if the guard, which may 
feature references to parameters, evaluates to true given the current param- 
eter values. 

Value-Passing: values may now be communicated between sequential compo- 
nents via activities. 

Additional features could have been added, but they would not have greatly 
strengthened the usefulness of the language. In the next section, we present 
a mapping from PEPA^ to PEPA, and illustrate that quite deliberately, the 
underlying algebra has not been changed. 

4.1 A Semantics for PEPAfc 

In this section, we provide a PEPA semantics for PEPAfc. We do this so that 
the behavior of PEPAfc models within Mobius can be understood formally, and 
it is important to note that this translation is not carried out within the tool 
itself. Before we give our semantics, we give a preliminary definition that is used 
to construct PEPA cooperation and hiding sets. 

Definition 2 (Refining data-passing activities) Let L be a set of action 
types, and T be the set of Mobius state variable types (from Section ed. li . The 
function refine(L) is defined as: 

L U {oi : a G L,i G T} 

As is conventional, we map all a-activities used in value passing to T-indexed a- 
activities. Due to the construction of T, all such refined cooperation and hiding 
sets remain countable. Now we can define a PEPA semantics for PEPAfc. 

Definition 3 (PEPA Semantics for PEPAfc) Let eval(e, E) represent the 
simple evaluation of expression e in an environment E. Then [.[e; is a func- 
tion that given an environment E mapping variables to values, maps a PEPAk 
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process or defining equation to a set of PEPA proeesses or defining equations as 
follows: 



lP[x] = Q\e = {Pi = \Q\e' :ii, . . . ,in &T,E' = E[ii/xi, . . . ,in/xn]} 
IP ^ QIe = Ip\e 

IP/L\e = lPli5/refine(L) 

(IPIe */101i=; = 0 

< lOliJ */l^^l£ = 0 

+ lOls otherwise 



[if b then P\e = 

|(a!e, r).Q\E = 
l(a?x, r).(3|£; = 



I I^’Ib f/eval(6, £;) = true 
1 0 otherwise 

(^eval(e,£J) ; (r, £/ ) ) . | Q | £; 

^ (ai,eval(r,£:)).|(3|£:[i/^] 

{a; : ieT} 



{a,r).Q\E = (o;,eval(r,£;)).|(5|£; 

l^[— ]|t? -^eval(ei ,eval(eTi ,i?) 



A guarded PEPA^ process P is mapped to P if the guard is true in the current 
environment; otherwise it is mapped to the deadlocked process. Deadlocked pro- 
cesses are then removed if they are found to appear in choice contexts. Of course 
it is still possible to write a deadlocked PEPA^ process, just as it is possible to 
write a deadlocked PEPA process. Parameterized process definitions are mapped 
to a set of indexed definitions for each possible combination of parameter val- 
ues, and process constants provide a way for the PEPA^ process to change the 
values of variables itself. A PEPA^ process of the form (a?x, r).P is mapped to 
a sum (choice) over all ai-guarded copies of P, for f G T (recall that the set T is 
arbitrarily large but finite). A PEPA^ process of the form {a\e,s).P is mapped 
to a single PEPA process, specifically {aj,s).P, where j is the result of evalu- 
ating e in the environment E. This means that a PEPAj, process of the form 
(a?x, r).P ^ (a!e, s).P would be capable of one transition, via a single activity 



of type Oj . This has the effect of setting the value of x in the left subcomponent 
to be equal to j. This scheme also means that (a!f , r).P ^ (a!e, s).P is dead- 
locked unless eval(e, E) — eval(/, E). This is reasonable, with the interpretation 
that if both subcomponents are trying to force the setting of a variable in an- 
other subcomponent, then they must agree on the value for the variable. This 
scheme has been used in previous parameterized process algebras; for example, 
see II 1 II 21 . 



Our chosen semantics is not suitable for understanding a PEPAj, process 
such as P {a?x,r).P' in isolation. As described in H3I, the choice over all 
Oi-guarded copies of P' means that the sojourn time in state P is not 1/r 
as would be expected, but rather decreases arbitrarily. However, this does not 
cause problems for our implementation. We insist that all PEPA^ processes 
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specified evolve such that if an input activity of type a is enabled, it must be 
within a cooperation context that enables an output activity of type a. If, at 
any point during its evolution, the PEPA^ process represented by the model’s 
system equation enables an unmatched input activity, then the model is in error. 
This should be considered in the same light as the specification of a model with a 
deadlock; it is perfectly possible to write an incorrect model specification. In our 
implementation, Mobius catches this error during model execution and halts. It 
can be shown that for any PEPA^ model satisfying the condition given above, 
our semantics lead to a PEPA model with an equivalent performance model. 
Alternative semantic models do exist for value-passing algebras; for example, the 
STGLA model |1 ,'Ij avoids branching problems by maintaining process variables 
and expressions in a symbolic form. 

Below we give a PEPA^ model of a simple M/M/s/n queue and illustrate 
the translation to PEPA. 

Example 1. 

Queue[m, s, n] if (m < n) then (in, X).Queue[m + 1 , s,n] 

-|- if (m > 0) then (out, fj, * min(s, m)).Queue[m — 1 , s,n] 

translates to a set of definitions over values of s and n, including the following: 
Queueo,s,n {±n, X). Queue i ^s,n 

Queuei^s,n (in, X).Queuei+i ^s,n + (out, p, * i).Queuei-i^s,n for 0 < i < s 

Queuei^s,n (in, X).Queuei+x ^s,n + (out, p * s).Queuei-x ^s,n for s < i < n 

Queuen,s,n (out,^ * m).Queue„-ps,n 

We have presented PEPA^, an extension to PEPA, and shown that while ex- 
pressiveness has been improved and additional modeling flexibility has been pro- 
vided, the underlying process algebra has not changed. In the next section, we 
present our application of PEPA^ to the identification of Mobius state variables. 



5 Mapping a PEPA^ Process to the AFI 

We present a practical technique for identifying state variables in an SPA model, 
and provide a formal mapping to the Mobius AFI. State variables will be given 
by PEPAf. process parameters. This means that the modeler will provide explicit 
guidance on exactly what state may be shared, and means that the modeler will 
create the PEPA model in such a way that the effect of a change in shared state 
will be well understood. In order to list the state variables of a PEPA^ process, 
we first provide the definition of an auxiliary function that calculates the state 
variables of a sequential PEPA^ process. 
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Definition 4 (State variables of a sequential PEPA^ process) The 

state variables of a sequential PEPAk proeess P are given by svars(P, 0) G X, 
where svars(., W) is defined as follows: 

svars(a.S', W) = svars(5, W) 
svars(S' + T, W) = svars(5, W) U svars( T, W) 



s{A[el,W) = 



svars(if b then S, W) — svars(5', W) 

({x} U svars(S', W U (A, n)) if A[x\ S and (A, n) ^ W 
0 otherwise 



This definition deliberately fails to distinguish between a parameter x of process 
S and another parameter with the same name of process T, if T G ds{S). We 
consider these to represent the same state variable. This mechanism allows the 
PEPAfe modeler to change the value of a state variable simply; for example, if S 
and T both have a parameter named x then T[x\ (a, r).S[f{x)] will perform 
an activity (a, r), and immediately afterwards change x's current value from a 
to /(a). Now we give the state variables for a PEPA^ process, P. 



Definition 5 (State variables of a PEPA^ process) Let 

P = {Si,... ,Sn)p be a PEPAk proeess in vector form. The state variables of 
P are given by: 

[J : X G svars(5j, 0)} 

l<2<n 

Each state variable is instrumented with an index according to its position in 
the vector, to ensure that duplicate names do not clash. This completes the 
definition of the AFI state variables. However, the state variables alone do not 
characterize the state of the PEPA^ process; it is necessary to take into account 
the current PEPAfc term too. Just as described in Section IQ the process state 
can be captured using a vector notation. This state is maintained for interacting 
with model solvers, but is not exported for use in state-sharing or the definition 
of performance variables. The complete state of a PEPAj, process is discussed 
further in Section l.b. 1 1 In order to complete the mapping to the AFI, we must 
generate from the PEPA^ process a set of Mobius actions. There are several ways 
in which this can be done. Given a PEPA^ process P = {Si , . . . , Sn)p, there are 
two possibilities for the AFI actions: 

— for every P' G ds{P), the enabled activities of P' (distinguishing duplicates). 
We reject this idea since it would require that we generate the entire state 
space of the underlying process in order to provide the mapping. 

— for 1 < z < n, for every S{ G ds{Si), the enabled activities of S' (distinguish- 
ing duplicates). This is reasonable, and can be implemented efficiently. One 
drawback is in the consideration of impulse rewards; if three sequential com- 
ponents cooperate over an activity, a reward could not be specified for the 
firing of an activity, as the result of a cooperation between a particular two 
of the three components. 
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Instead, we chose a method that can be efficiently computed and does not sac- 
rifice expressibility. Definition 0 specifies the type of Mobius actions. 



Definition 6 (Set of Actions) The set MA is the least set inductively defined 
as follows: 

— (a, P) G MA for all activities a and PEPAk processes P 

— if B G MA then (a, P, B) G MA 

— if Bi, B2 G MA then (a, P, Bi l±l B2) G MA 

Now let P be a PEPA^, process. The API actions of P are given by 
actions(P,0) G MA, where actions(., bP) is defined as follows: 



Definition 7 (Actions for a PEPA^ process) Let P be a PEPAk process. 
The API actions of P are given by actions(P, 0), where actions(., VP) is defined 
as follows: 



actions(S' -I- T, W) = actions(5', W) U actions( T, W) 
actions((o:?x, r).S, W) = {((a, r), S)} U actions(5', W) 
actions((o;!e, r).S, W) = {((a, r), S)} U actions(5', W) 
actions((a, r).S, W) = {((a, r), S)} U actions(6', W) 
actions(if b then S, W) = actions(5', W) 



actions(A[e], TP) = 



j actions(P, W U {A, n)) if A[x\ = P and {A, n) 

otherwise 

let T'p — actions(P, W) and L>q = actions(<5, W); then 
actions(P Q, W) = {((a, r),P' ^ Q,B)\B = ((a, r), P', A) Gd>p,a ^ L}A 
{((a, r),P \>fi Q' , B) ■. B = {{a,r), Q',A) S <?Q,a ^ L} U 



{((o, R),P' ^ Q', Pi W B2 ) : (a, r).P' ^ (a, s).Q'^, 
Pi = {{a,r),P',Ai) G ^p, 

P 2 = ((a,s), Q',A 2 ) G -Pq} 

actions(P/P, W) = {((r, r),P' , B) : B = ((a, r),P' , A) GPp,a G L}\J 
{{{a,r),P' ,B)\B = {{a,r),P,A) G Pp,a ^ L} 



For each derivative of each sequential component, this function computes every 
enabled activity. The API actions of a cooperation P \>^ Q are 

— actions associated with individual behavior of each subcomponent (types 
that do not match those in L). 

— for each pair of API actions with types a G L, a, new action consisting of a 
cooperation between the two. 

Thus a Mobius action a is a structure consisting of a PEPA activity, the deriva- 
tive that results from its completion, and then a set of the Mobius actions of the 
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model’s subcomponents that have combined to form a. The advantage of this 
technique is that every activity that could be enabled by any derivative of P 
is mapped to a Mobius AFI action. This means that the Mobius modeler can 
distinguish and assign an impulse reward to a composite activity resulting from 
the evolution of a chosen subset of model subcomponents. The disadvantage of 
this technique is that since we directly compute “products” of activities, there 
may be a proliferation of AFI actions that correspond to PEPA activities that 
may never be enabled. 

We have presented a mapping from PEPA^ to the Mobius AFI. This means 
that PEPAfc models can be composed with other Mobius atomic models using 
equivalence sharing. The ability of another model to unilaterally change some 
shared state has some consequences for the parameterized process, which we 
discuss next. 

5.1 Implications of Modeling with PEPAj. 

We have described a mapping from PEPA^ to the AFI, and have shown that 
PEPAfe is no more powerful than PEPA itself. By providing the modeler with 
some of the convenience of a programming language, we 

— facilitate the construction of concise PEPA/^ specifications that have a nat- 
ural PEPA semantics. 

— cause the modeler to structure his definitions in such a way that a change 
in the value of a state variable (PEPAj, process parameter) due to a partner 
model will cause a meaningful and understandable change in the state of the 
PEPAfc model. 

The last point is an important one, and justifies our selection of this method 
for implementation. The addition of such “cues” into the PEPA model makes 
equivalence sharing meaningful and useful. 

In Section 0 we stated that the PEPA formalism maps process parameters 
to state variables for use in state sharing and the definition of performance 
variables, but that the state of the PEPA^ process is also maintained and com- 
municated to model solvers via the AFI. This means that we can construct a 
PEPAfe model P that Mobius interprets as having a larger state space than 1P[0. 
Consider the following PEPA^ definition: 

5[a;] if a: yf 1 then (ai, r).(/3, s).-?)!] 

-I- if a: yf 2 then (tt 2 , r).(/3, s).-S'[2] 

-I- if a: yf 3 then (as, r).(/3, s).S'[3] 

Translating this to PEPA leads to: 

51 ( 02 , r).(/3, s).S 2 + {as, r).(/3, s).S 3 

5 2 (oi, r).(/3, s).Si + {as, r).{(3, s).S 3 

5 3 {ai,r).{P, s).Si + (« 2 , r).{P, s).S 2 
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The PEPA process has 6 states. However, using the API, the Mobius state space 
generator will detect 9 unique states for the PEPA^ process. The reason for this 
is that the process itself can be in the state (/3, s).5'[l] while the state variable x 
may either have the value 2 or the value 3 (and similarly for the other derivatives 
of 5[x]). These states are equal by every reasonable process algebra equivalence, 
since (/3, s).5'[l] makes no further use of the value of x. However, what is crucially 
different is that the AFI now allows a partner model to use the value of x while 
the PEPAfe model is in this state. If the modeler wishes to generate the smallest 
reasonable state space, the model could alternatively be specified as below: 

5[x] if X yf 1 then (ai, 

+ if X yf 2 then (a 2 , r).S'[2] 

+ if X yf 3 then (as, r).5’'[3] 

5"[x] =*' (/3,s).5[x] 

This works because the value of the state variable is changed one activity sooner. 
However, this process will not behave identically to its original partner process. 
Furthermore, although in isolation, the PEPA^ model has a state space consis- 
tent with the translated PEPA model. In an equivalence sharing relationship, 
the partner model still has the opportunity to change the value of x at any point, 
leading in this case to a slightly larger state space. If the modeler aims to employ 
his PEPAfc model in an equivalence sharing relationship, this is a novel issue of 
which he must be aware. 

One interesting aspect of this work is the extent to which equivalence relations 
may be employed for aggregation when equivalence sharing is being used. We 
can certainly ensure that P P' is treated as equivalent to P' P and check 
for this as detailed in However, if P is defined with a parameter, e.g., 

P[x] (a, r).P'[x -I- 1], then P[a] ^ P'[b] will export two state variables, xi 
and X 2 , with values a and b respectively, via the AFI. A partner model may be 
relying on the individual values of these variables, and thus it would be incorrect 
to equate this process to P'[6] P[a]. For processes with parameters, the order 

in which they appear in the static structure of a term must be preserved. 

6 Example 

We present an example to illustrate the implementation and use of PEPA^ in 
Mobius. Our model is of a simple factory system that consists of three robot 
arms. Two robots are responsible for removing items from a conveyor belt, pack- 
aging several items together into two units and then depositing one assembled 
unit into each of the two bins. The third robot removes assembled units from 
the shared bins and places them on another conveyor belt for later processing. 
It does this by always attempting to remove an item from the first bin if it can, 
and only if it cannot, removing and passing along an item from the second bin. 

The first two robot arms behave identically, and are modeled by one SAN 
as shown in Figure!^. The leftmost activity fires until place Accumulate is full; 
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meanwhile, the robot arm assembles units, incrementing the values of places Binl 
and Bin2 when activity Assemble fires. The PEPA^ model of the third robot arm 
is shown in FigureEb- From the specification of process Consume[bl, 62], it can be 
seen that activity (outbl, blr) is enabled if the value of parameter 61 is positive, 
and (outb2, b2r) is enabled if 62 is positive, and 61 equals zero. From here, the 
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Fig. 2. Example models 

overall model of the system can be easily built, and is shown in Figure Efc. We 
use the Replicate/Join formalism; two copies of the first robot are produced by 
applying a replicate node to the SAN model. The Replicate creates an integer 
number of copies of a model, and does not distinguish individual copies. We 
insist that both copies of the first robot share places Binl and Bin2. Next we 
use a Join node to apply equivalence sharing to the Replicated model of the first 
robot, and the PEPAfc model of the second robot. The Mobius tool allows us 
to specify that shared place Binl should be further shared (identified) with the 
parameter 61 of the PEPA model, and similarly that Bin2 should be identified 
with 62. In this way, we create an accurate model of the system as a whole. Due 
to our mapping to the AFI, it is now possible to use the Mobius tool’s analytical 
solvers or discrete-event simulator to investigate the behavior of this model over 
time. 

7 Future Work and Conclusion 

One area of future work is in the design of an action-sharing composition mod- 
eling formalism for Mobius. Here, Mobius can benefit from the experience and 
results of the SPA community. PEPA, and SPAs in general, are packaged with 
a compositional theory based around synchronization via actions. In contrast, 
the Mobius framework, and its implementation as the Mobius tool, currently 
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supports the synchronization of concurrent models via shared state. One reason- 
able and useful extension to the theory would be to allow action-synchronization 
operators, such as PEPA’s cooperation, to be applicable to any submodels that 
satisfy the API. Many choices of operator may be useful; for example, the op- 
erator may insist that some particular subset of “cooperating” activities com- 
plete before the submodels change state. The literature features several ways in 
which the rate of cooperation can be chosen IS]. Furthermore, there is work on 
building compositional Petri net-based modeling languages PHUH, and also on 
exploiting process algebra results in the development of a compositional Petri 
net theory m- By combining such action-based operators with Mobius’s Repli- 
cate/Join composition formalism, it will be possible to create a new and general 
model composition formalism. Furthermore, both formalisms provide support 
for state space aggregation based upon identifying “replicated” subcomponents, 
and this aggregation should be preserved in any joint formalism. With the po- 
tential to communicate data between submodels using actions, and with current 
work on exploiting group theory to detect symmetries this new formalism 
has the potential to be fruitful in both theory and practice. 

We have presented a method of allowing PEPA models to be composed with 
other models via equivalence sharing, and thus for incorporating PEPA into 
the Mobius tool. As a result, the Mobius tool may now be used to specify and 
solve PEPA models, and to combine them with existing modeling formalisms, 
such as SANs and an extended Markov chain formalism called Buckets and 
Balls |2n| • We believe this illustrates the flexibility and generality of both the 
Mobius framework and the AFI. Furthermore, we expect that a similar mapping 
can be developed for SPAs other than PEPA. 
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